Can we start using C++ STL containers in Mozilla code?
Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? Many of the STL containers are more convenient to use than our equivalents, and more familiar to new contributors. I understand that we used to have a policy of not using STL in mozilla code since some older compilers we wanted to support didn't have very good support, but I'd assume that that argument no longer holds since already build and ship a bunch of third party code that uses std containers (angle, webrtc, chromium IPC, crashreporter), and the sky hasn't fallen. I'm not proposing a mass rewrite converting nsTArray to std::vector, just that we allow STL in new code. Are there valid reasons why should we not allow C++ STL containers in Mozilla code? Cheers, Chris P. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
Note that we already do use at least those STL containers for which we don't have an equivalent in the tree. I've seen usage of at least: std::map, std::set, and std::bitset. I think that Nick has a good point about reporting memory usage, but I think that the right solution to this problem is to add Mozilla equivalents for the STL data structures that we need to fork, not to skew all your design to use the data structures that we have instead of the ones you need. Forking STL data structures into Mozilla code seems reasonable to me. Besides memory reporting, it also gives us another benefit: guarantee of consistent implementation across platforms and compilers. Benoit 2013/12/10 Nicholas Nethercote n.netherc...@gmail.com On Tue, Dec 10, 2013 at 8:28 PM, Chris Pearce cpea...@mozilla.com wrote: Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code#C.2B.2B_and_Mozilla_standard_libraries has the details. As a general rule of thumb, prefer the use of MFBT or XPCOM APIs to standard C++ APIs. Some of our APIs include extra methods not found in the standard API (such as those reporting the size of data structures). I'm particularly attuned to that last point. Not all structures grow large enough to be worth reporting, but many are. In the past I've converted STL containers to Mozilla containers just to get memory reporting. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
Also note that IIUC, the only thing that prevents us from solving the memory-reporting problem using a STL allocator, is that the spec doesn't allow us to rely on storing per-object member data on a STL allocator. Even without that, we could at least have a STL allocator doing per-STL-container-class memory reporting, so that we can at least know how much memory is taken by all std::set 's together. Just so we know if that ever becomes a significant portion of dark matter. Benoit 2013/12/10 Benoit Jacob jacob.benoi...@gmail.com Note that we already do use at least those STL containers for which we don't have an equivalent in the tree. I've seen usage of at least: std::map, std::set, and std::bitset. I think that Nick has a good point about reporting memory usage, but I think that the right solution to this problem is to add Mozilla equivalents for the STL data structures that we need to fork, not to skew all your design to use the data structures that we have instead of the ones you need. Forking STL data structures into Mozilla code seems reasonable to me. Besides memory reporting, it also gives us another benefit: guarantee of consistent implementation across platforms and compilers. Benoit 2013/12/10 Nicholas Nethercote n.netherc...@gmail.com On Tue, Dec 10, 2013 at 8:28 PM, Chris Pearce cpea...@mozilla.com wrote: Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code#C.2B.2B_and_Mozilla_standard_libraries has the details. As a general rule of thumb, prefer the use of MFBT or XPCOM APIs to standard C++ APIs. Some of our APIs include extra methods not found in the standard API (such as those reporting the size of data structures). I'm particularly attuned to that last point. Not all structures grow large enough to be worth reporting, but many are. In the past I've converted STL containers to Mozilla containers just to get memory reporting. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/2013 11:28 AM, Chris Pearce wrote: Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? Many of the STL containers are more convenient to use than our equivalents, and more familiar to new contributors. I understand that we used to have a policy of not using STL in mozilla code since some older compilers we wanted to support didn't have very good support, but I'd assume that that argument no longer holds since already build and ship a bunch of third party code that uses std containers (angle, webrtc, chromium IPC, crashreporter), and the sky hasn't fallen. I'm not proposing a mass rewrite converting nsTArray to std::vector, just that we allow STL in new code. Are there valid reasons why should we not allow C++ STL containers in Mozilla code? Cheers, Chris P. std::map/set may not have the performance characteristics people think. They are significantly slower than xpcom hashtables (since they are usually trees), yet people tend to use them as HashTables or HashSets. Someone should compare the performance of std::unordered_map to our hashtables. (Problem is that the comparison needs to be done on all the platforms). -Olli ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/13 9:49 AM, Benoit Jacob wrote: since AFAIK we don't have a HashSet class in mfbt or xpcom. It's called nsBaseHashtable. Granted, using it as a hashset is not that intuitive. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On Tue, Dec 10, 2013 at 1:28 AM, Chris Pearce cpea...@mozilla.com wrote: Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? Many of the STL containers are more convenient to use than our equivalents, and more familiar to new contributors. I understand that we used to have a policy of not using STL in mozilla code since some older compilers we wanted to support didn't have very good support, but I'd assume that that argument no longer holds since already build and ship a bunch of third party code that uses std containers (angle, webrtc, chromium IPC, crashreporter), and the sky hasn't fallen. I'm not proposing a mass rewrite converting nsTArray to std::vector, just that we allow STL in new code. Are there valid reasons why should we not allow C++ STL containers in Mozilla code? Cheers, Chris P. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform One thing I haven't seen mentioned yet are the codesize consequences of using STL containers. Last time I saw numbers the STL containers generated significantly more code than their XPCOM equivalents. - Kyle ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/13 9:09 AM, Boris Zbarsky wrote: On 12/10/13 9:49 AM, Benoit Jacob wrote: since AFAIK we don't have a HashSet class in mfbt or xpcom. It's called nsBaseHashtable. Granted, using it as a hashset is not that intuitive. There's also js::HashSet in js/public/HashTable.h. -j ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/2013 3:28 AM, Chris Pearce wrote: Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? Many of the STL containers are more convenient to use than our equivalents, and more familiar to new contributors. I understand that we used to have a policy of not using STL in mozilla code since some older compilers we wanted to support didn't have very good support, but I'd assume that that argument no longer holds since already build and ship a bunch of third party code that uses std containers (angle, webrtc, chromium IPC, crashreporter), and the sky hasn't fallen. The sizeOfExcludingThis/sizeOfIncludingThis methods on our containers are one feature not found in STL. There are some codesize concerns in using STL, as it is much more template-heavy than our own implementations (particularly in regards to std::unordered_map/nsTHashtable). The final issue is that performance characteristics may be very different: C++11 disallows copy-on-write semantics for std::string, for example; in another edge case, most STL containers' move constructors require noexcept semantics to become enabled (and autodegrades move to copy in most cases), and I'm not certain that compiling without exception support would grant us those semantics. Also, std::set and std::map are the wrong containers to use 95% of the time. If people don't want to use nsTHashtable because the API sucks, it's better to change the API for nsTHashtable rather than use the wrong datatype. I think the right answer is to make the myriad of datatypes we have more STL-ish--as a first step at least, can we at least add iterator support for them and get rid of the abominable EnumerateForwards/EnumerateBackewards methods? This would allow us to use range-based for once our minimum compiler support gets there. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/2013 9:49 AM, Benoit Jacob wrote: std::set compared to using a hashtable to do the same thing, since AFAIK we don't have a HashSet class in mfbt or xpcom. nsTHashtableKeyType is the XPCOM hashset. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should we disable autoplay feature of HTMLMediaElement on mobile?
On 09/12/13 08:44 PM, Tetsuharu OHZEKI wrote: For mobile, this delaying approach is well for saving power. I feel this approach make sense. Power? How about not blowing through your data cap allowance or paying and other hefty charges Very few countries have cheap unlimited data over cellular. Hub ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should we disable autoplay feature of HTMLMediaElement on mobile?
On 08/12/13 04:49 AM, Robert O'Callahan wrote: Don't these arguments apply to desktop Firefox used at work, in an Internet cafe, or in a library, as well? I think it's important to have an easy way to mute/unmute the browser, but disabling autoplay is probably not the right way to address these issues. It should be noted that here on desktop I do have `media.autoplay.enabled=false` (I hate videos that autoplay) and this still doesn't stop YouTube. [1] So while I'm all for disabling video and sound autoplay across the board, that config flag isn't enough to get rid of that nuisance. [2] I'm not sure if it is a bug or a feature, but if you tell me it is a bug, then I'll gladly file one. Hub [1] before I get told CTP, I don't have Flash, so this is only for the case the video is playable without Flash. [2] I have already noted the disagreement of opinion here. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/2013 4:28 AM, Chris Pearce wrote: Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? Many of the STL containers are more convenient to use than our equivalents, and more familiar to new contributors. njn already mentioned the memory-reporting issue. Also, some of the containers are not suitable for use in a codebase that doesn't allow exceptions. For instance std::map.at() wouldn't be appropriate in our code and may not compile. AIUI, the current status of using the C++ standard containers is that they aren't necessarily forbidden, but we should be careful about checking our assumptions about how they use exceptions, how much they cost in codesize, and whether we need to reimplement or subclass them in MFBT in order to get better memory reporting. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Bookmarks and history
Hello, I wanted to know if there is any way to access bookmarks and history from the firefox mobile sdk like the places api for the desktop version. Another question, when I try to cfx run with my firefox mobile, the firefox crashes and doesn't work. And I tried with something simple, like open a tab and firefox crashed with a simple App like that. Thanks a lot, Shweta. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
MemShrink Meeting - Tuesday, 10 December 2013 at 2:00pm PST
The next MemShrink meeting will be brought to you by the Nuwa process on B2G: https://bugzilla.mozilla.org/show_bug.cgi?id=930282 The wiki page for this meeting is at: https://wiki.mozilla.org/Performance/MemShrink Agenda: * Prioritize unprioritized MemShrink bugs. * Discuss how we measure progress. * Discuss approaches to getting more data. Meeting details: * Tue, 10 December, 2:00 PM PST * http://arewemeetingyet.com/Los%20Angeles/2013-12-10/14:00/MemShrink%20Meeting * Vidyo: Memshrink * MTV: Very Good Very Mighty, Mountain View office, 3rd floor * SF: Noise Pop. 7th Floor * Dial-in Info: - In office or soft phone: extension 92 - US/INTL: 650-903-0800 or 650-215-1282 then extension 92 - Toll-free: 800-707-2533 then password 369 - Conference num 98802 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Disable Windows builds and nightly builds on b2g18 and b2g18-v1.0.0-hd
As I was looking into the patches I noticed that if we remove the Windows tests I would not be able to find a good reason to trigger the win32opt debug builds as they would be triggering nothing. I will be disabling the win32 builds as well. Please comment in the bug in case you have any concerns about it: https://bugzilla.mozilla.org/show_bug.cgi?id=948135 cheers, Armen On 13-12-09 01:52 PM, Armen Zambrano G. wrote: Changing the subject to make it more clear. I will be raising this on the Monday weekly call as well as the Engineering meeting. On 13-12-07 03:31 PM, Armen Zambrano G. wrote: (Please follow up on mozilla.dev.b2g) Adding dev.platform to reach a wider audience. I will bring this up at the Engineering meeting on Tuesday in case I don't hear anything back by Tuesday. cheers, Armen On 13-12-06 02:50 PM, Armen Zambrano G. wrote: Hi all, Given that we are only doing security fixes on those two branches. Anyone can say if we still need desktop unit tests and nightly builds on those two branches? [1][2] Would running B2G builds/tests and linux32 build/tests be good enough? Those jobs are running on the old Rev3 minis and would be great to get rid of them now rather than wait until March. regards, Armen [1] https://tbpl.mozilla.org/?tree=Mozilla-B2g18 [2] https://tbpl.mozilla.org/?tree=Mozilla-B2g18-v1.1.0hd ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/2013 09:49 AM, Benoit Jacob wrote: std::map and std::set are also free of any size restriction, and do not require a contiguous part of our address space (see https://bugzilla.mozilla.org/show_bug.cgi?id=944810#c12 ) , so they won't run OOM before we are actually OOM. I posit that if there is a map/set where the number of entries causes this to be a concern, we should not be using types where we can't report the memory usage. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
It seems to me that we should be optimizing for developer productivity first, and use profiling tools to find code that needs to be optimized. i.e. we should be able to use STL containers where we need basic ADTs in day-to-day coding, and if instances of these containers show up in profiles then we should look at moving indivdual instances over to more optimized data structures. On 12/11/2013 4:42 AM, Benjamin Smedberg wrote: njn already mentioned the memory-reporting issue. We already have this problem with third party libraries that we use. We should work towards having a port of the STL that uses our memory reporters, so that we can solve this everywhere, and influence the size of generated code for these templates. Also, some of the containers are not suitable for use in a codebase that doesn't allow exceptions. For instance std::map.at() wouldn't be appropriate in our code and may not compile. We compile with -D_HAS_EXCEPTIONS=0 on Windows, so the use of functions that throw should be caught at compile time, or at worst cause backouts at landing. Won't that prevent us throwing exceptions? AIUI, the current status of using the C++ standard containers is that they aren't necessarily forbidden, but we should be careful about checking our assumptions about how they use exceptions, how much they cost in codesize, and whether we need to reimplement or subclass them in MFBT in order to get better memory reporting. Based on https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code#C.2B.2B_and_Mozilla_standard_libraries (thanks njn!) we can use std::unordered_map and std::unordered_set at least. That still won't give us a convenient to use queue/deque through. Chris P. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On Wed, Dec 11, 2013 at 10:29 AM, Chris Pearce cpea...@mozilla.com wrote: It seems to me that we should be optimizing for developer productivity first, and use profiling tools to find code that needs to be optimized. i.e. we should be able to use STL containers where we need basic ADTs in day-to-day coding, and if instances of these containers show up in profiles then we should look at moving indivdual instances over to more optimized data structures. Keep in mind that proliferation of different types for the same functionality hurts developer productivity in various ways, especially when they have quite different APIs. That's the main reason I'm not excited about widespread usage of a lot of new (to us) container types. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
2013/12/10 Chris Pearce cpea...@mozilla.com It seems to me that we should be optimizing for developer productivity first, and use profiling tools to find code that needs to be optimized. i.e. we should be able to use STL containers where we need basic ADTs in day-to-day coding, and if instances of these containers show up in profiles then we should look at moving indivdual instances over to more optimized data structures. On 12/11/2013 4:42 AM, Benjamin Smedberg wrote: njn already mentioned the memory-reporting issue. We already have this problem with third party libraries that we use. We should work towards having a port of the STL that uses our memory reporters, so that we can solve this everywhere, and influence the size of generated code for these templates. I agree with the above. I would also like to underline an advantage of the STL's design: the API is very consistent across containers, which allows to most easily switch containers (e.g. switch between map and unordered_map) and recompile. This has sometimes been derided as a footgun as one can unintentionally use a container with an algorithm that isn't efficient with it. But this also has a really nice, important effect: that one can avoid worrying too early about optimization details, such as whether a binary tree is efficient enough for a given use case or whether a hash table is needed instead. By contrast, our current Mozilla containers have each their own API and no equivalent of the STL's iterators, so code using one container becomes married to it. I believe that this circumstance is why optimization details have been brought up IMHO prematurely in this thread, needlessly complicating this conversation. 2013/12/10 Robert O'Callahan rob...@ocallahan.org Keep in mind that proliferation of different types for the same functionality hurts developer productivity in various ways, especially when they have quite different APIs. That's the main reason I'm not excited about widespread usage of a lot of new (to us) container types. For the same reason as described above, I believe that adopting STL containers is the solution, not the problem! The STL shows how to design containers that have a sufficiently similar API that, in most cases where that makes sense (e.g. between a map and an unordered_map), you can switch containers without having to adapt to a different API. Benoit ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
Also note that IIUC, the only thing that prevents us from solving the memory-reporting problem using a STL allocator, is that the spec doesn't allow us to rely on storing per-object member data on a STL allocator. I believe C++11 supports this. Cheers, Botond ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On Tue, Dec 10, 2013 at 09:17:22AM -0600, Joshua Cranmer ? wrote: On 12/10/2013 3:28 AM, Chris Pearce wrote: Hi All, Can we start using C++ STL containers like std::set, std::map, std::queue in Mozilla code please? Many of the STL containers are more convenient to use than our equivalents, and more familiar to new contributors. I understand that we used to have a policy of not using STL in mozilla code since some older compilers we wanted to support didn't have very good support, but I'd assume that that argument no longer holds since already build and ship a bunch of third party code that uses std containers (angle, webrtc, chromium IPC, crashreporter), and the sky hasn't fallen. The sizeOfExcludingThis/sizeOfIncludingThis methods on our containers are one feature not found in STL. There are some codesize concerns in using STL, as it is much more template-heavy than our own implementations (particularly in regards to std::unordered_map/nsTHashtable). The final issue is that performance characteristics may be very different: C++11 disallows copy-on-write semantics for std::string, for example; in another edge case, most STL containers' move constructors require noexcept semantics to become enabled (and autodegrades move to copy in most cases), and I'm not certain that compiling without exception support would grant us those semantics. Also, std::set and std::map are the wrong containers to use 95% of the time. If people don't want to use nsTHashtable because the API sucks, it's better to change the API for nsTHashtable rather than use the wrong datatype. I think the right answer is to make the myriad of datatypes we have more STL-ish--as a first step at least, can we at least add iterator support for them and get rid of the abominable EnumerateForwards/EnumerateBackewards methods? This would allow us to use range-based for once our minimum compiler support gets there. +1 Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On Wed, Dec 11, 2013 at 10:50 AM, Benoit Jacob jacob.benoi...@gmail.comwrote: 2013/12/10 Robert O'Callahan rob...@ocallahan.org Keep in mind that proliferation of different types for the same functionality hurts developer productivity in various ways, especially when they have quite different APIs. That's the main reason I'm not excited about widespread usage of a lot of new (to us) container types. For the same reason as described above, I believe that adopting STL containers is the solution, not the problem! The STL shows how to design containers that have a sufficiently similar API that, in most cases where that makes sense (e.g. between a map and an unordered_map), you can switch containers without having to adapt to a different API. I'm not a fan of the STL's approach of making every data structure feel like an array. It leads to nonsense like the erase-remove idiom. A remove function that doesn't actually remove anything, now that's a footgun :-). Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Valgrind-on-TBPL
On Tue, Dec 3, 2013 at 5:02 PM, Nicholas Nethercote n.netherc...@gmail.com wrote: I'm working on getting Valgrind (Linux64-only) runs visible on TBPL. In order to understand what needs to be done, I looked at the Requirements for being shown in the default TBPL view, from From https://wiki.mozilla.org/Sheriffing/Job_Visibility_Policy. 3) Runs on mozilla-central and all trees that merge into it 4) Scheduled on every push Chris Atlee kindly implemented the required buildbot configuration changes, so the Valgrind runs now occur on every push to m-c, m-i, fx-team, etc. This is great. Unfortunately, on December 5th, just before the changes were enabled, something else happened that caused the Valgrind jobs to start timing out all the time. This is being tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=948145. The Valgrind-on-TBPL jobs do an --enable-valgrind build (using the mozconfig at browser/config/mozconfigs/linux64/valgrind), and then run scripts/valgrind/valgrind.sh (from the tools/ repo), which invokes the |make pgo-profile-run| target under Valgrind, which runs build/pgo/profileserver.py. At some point during the |make pgo-profile-run| target, we get this: command timed out: 1200 seconds without output, attempting to kill Because the Valgrind invocation is defined via the valgrind.sh script which is within the tools/ repo, instead of mozilla-central/, I've been able to do some interesting experiments -- I can alter the Valgrind invocation in valgrind.sh and re-trigger old jobs. Here's what I have learned. - It's not Valgrind that's causing the problem. If I change valgrind.sh to not invoke Valgrind at all, but instead run the test natively, I still get the timeouts. Here's sample output: + make pgo-profile-run /builds/slave/m-cen-l64-valgrind-000/objdir/_virtualenv/bin/python . ./src/build/pgo/profileserver.py command timed out: 1200 seconds without output, attempting to kill - Something changed on Dec 5. In the 3am job on Dec 5 on m-c, if I re-run the test under Valgrind it succeeds. If I re-run it natively, it times out. I've done numerous repeats and the behaviour is reliable. - In the 3am job on Dec 6 on m-c (the next available run), it times out consistently when I re-run it under Valgrind *or* natively. So something was fishy but it wasn't manifesting under Valgrind runs... and then something changed on Dec 5 that caused it to manifest all the time. The regression range is http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=1401e4b394adtoch ange=42b2a2adda8f - The timeout sometimes happens very early -- when running under Valgrind, it sometimes happens before Valgrind's start-up message is even printed. But sometimes it happens later. - When a job times out, the output produced by Valgrind is usually chopped off in the middle of a line. In fact, it's often chopped off in the middle of a line that would have been produced by a single write() system call. So it feels like the test harness is somehow abruptly stopping its recording of the output. - The runs that timeout take about 20 minutes longer to complete than those that don't. So the timeout-detection appears to be working correctly -- something is genuinely hanging for 20 minutes, AFAICT. I'd love to hear any ideas people might have about this, esp. relevant changes that landed on Dec 5. I have access to a Linux build slave but I'm not yet sure what I'll do with it. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: mozmake users, upgrade
Is this the default right now? If someone follows the steps and gets the Mozilla Build package on that page[1], will their build fail or succeed? [1]: https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions/Windows_Prerequisites ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
On 12/10/2013 3:29 PM, Chris Pearce wrote: It seems to me that we should be optimizing for developer productivity first, and use profiling tools to find code that needs to be optimized. For many of our developers, the STL is not an API that they are intimately familiar with, so it's not altogether clear that using the STL instead optimizes for developer productivity. Particularly since moving between the STL API and Mozilla ADT API is painful due to present differences, I don't think there is much benefit towards using STL directly. It's also worth noting that large parts of the STL API are horrendously misdesigned and have not been imitated by successor languages--std::map, std::iterator are some good examples here--reviewers have to be aware of the pitfalls of much of the STL, and I doubt many of them are aware of them (I sure am not). As an example here, when is std::vector::iterator invalidated by a mutation? What about std::map::iterator? What about std::unordered_map::iterator? (Hint: the answer is different for all three of them). And the number of STL implementations we use that even attempt to provide a debug mode to catch errors like this is 1, and it doesn't do a very thorough job of it. Using the correct data structure from the start isn't premature optimization: it's common sense, and we'd be killed by death of a thousand papercuts if we don't follow that rule. Also, some of the containers are not suitable for use in a codebase that doesn't allow exceptions. For instance std::map.at() wouldn't be appropriate in our code and may not compile. We compile with -D_HAS_EXCEPTIONS=0 on Windows, so the use of functions that throw should be caught at compile time, or at worst cause backouts at landing. Won't that prevent us throwing exceptions? I believe that in our current setup, code that attempts to throw an exception will compile but instead abort at runtime. AIUI, the current status of using the C++ standard containers is that they aren't necessarily forbidden, but we should be careful about checking our assumptions about how they use exceptions, how much they cost in codesize, and whether we need to reimplement or subclass them in MFBT in order to get better memory reporting. Based on https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code#C.2B.2B_and_Mozilla_standard_libraries (thanks njn!) we can use std::unordered_map and std::unordered_set at least. That still won't give us a convenient to use queue/deque through. Do you not see the giant note at the top of that page saying Don't trust this information? As the person who wrote 95% of that page, I can tell you that the STL section does not contain any information about what can and can't be used in Mozilla code, largely as a result of trying to balance the need to explain what can be used in Mozilla versus the desire to list the minimum features in C++11 code. This is particularly true for the ADT section, as the executive decision has been repeatedly affirmed to avoid STL containers as much as possible in Mozilla code. Note, for example, that STLport (the version we have in the tree) provides std::tr1::unordered_map, not std::unordered_map. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
Erase-remove seems like poor naming, more than anything. I think that iterators (pretending things are arrays) are generally really handy. Alternatives (needing to do everything with pfns) really tend towards arcaneness and are often awkward to use. In my experience, docs for 'our' containers are also miles behind what the STL has available. We don't *always* need to use the maximally efficient container type. Let profiling and knowledge of use-cases show us when we need fast containers. Most of the time, I just want simple containers. (particularly for DEBUG-only code) -Jeff - Original Message - From: Robert O'Callahan rob...@ocallahan.org To: Benoit Jacob jacob.benoi...@gmail.com Cc: Chris Pearce cpea...@mozilla.com, dev-platform dev-platform@lists.mozilla.org Sent: Tuesday, December 10, 2013 2:04:06 PM Subject: Re: Can we start using C++ STL containers in Mozilla code? On Wed, Dec 11, 2013 at 10:50 AM, Benoit Jacob jacob.benoi...@gmail.comwrote: 2013/12/10 Robert O'Callahan rob...@ocallahan.org Keep in mind that proliferation of different types for the same functionality hurts developer productivity in various ways, especially when they have quite different APIs. That's the main reason I'm not excited about widespread usage of a lot of new (to us) container types. For the same reason as described above, I believe that adopting STL containers is the solution, not the problem! The STL shows how to design containers that have a sufficiently similar API that, in most cases where that makes sense (e.g. between a map and an unordered_map), you can switch containers without having to adapt to a different API. I'm not a fan of the STL's approach of making every data structure feel like an array. It leads to nonsense like the erase-remove idiom. A remove function that doesn't actually remove anything, now that's a footgun :-). Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
For many of our developers, the STL is not an API that they are intimately familiar with, so it's not altogether clear that using the STL instead optimizes for developer productivity. Developers come and go, however, and the average C++ developer is more likely to be familiar with STL interfaces than with Mozilla ADT interfaces. It's also worth noting that large parts of the STL API are horrendously misdesigned and have not been imitated by successor languages--std::map, std::iterator are some good examples here-- Templates are a feature of C++ that have not been imitated by successor languages (no, generics do not count), and yet they are one of the most powerful features of C++. I view the STL's iterator model in a similar way. As an example here, when is std::vector::iterator invalidated by a mutation? What about std::map::iterator? What about std::unordered_map::iterator? (Hint: the answer is different for all three of them). While the answers are different for all of them, they (1) follow logically from how the data structures are expected to be implemented and (2) are well-documented. Cheers, Botond ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Valgrind-on-TBPL
Nicholas Nethercote writes: - When a job times out, the output produced by Valgrind is usually chopped off in the middle of a line. In fact, it's often chopped off in the middle of a line that would have been produced by a single write() system call. So it feels like the test harness is somehow abruptly stopping its recording of the output. Yes, sounds like all processes in the test are being killed. Most tests have a shorter time out to kill with signals that generate stacks, before this higher level time out kicks in. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should we disable autoplay feature of HTMLMediaElement on mobile?
2013/12/11 Hubert Figuière h...@mozilla.com: Power? How about not blowing through your data cap allowance or paying and other hefty charges Very few countries have cheap unlimited data over cellular. Of course, it includes network data traffic. I hope it saves power/data cost :) -- Tetsuharu OHZEKI saneyuki.s.s...@gmail.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should we disable autoplay feature of HTMLMediaElement on mobile?
2013/12/11 Hubert Figuière h...@mozilla.com: It should be noted that here on desktop I do have `media.autoplay.enabled=false` (I hate videos that autoplay) and this still doesn't stop YouTube. [1] I seem that Gecko's implementation of HTMLMediaElement.play() doesn't handle `media.autoplay.enabled`. http://mxr.mozilla.org/mozilla-central/source/content/html/content/src/HTMLMediaElement.cpp#2112 -- Tetsuharu OHZEKI saneyuki.s.s...@gmail.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Can we start using C++ STL containers in Mozilla code?
[I'm highlighting this exchange, because it's been buried somewhat; apologies for the misleading information] On Tue, Dec 10, 2013 at 4:29 PM, Joshua Cranmer pidgeo...@gmail.com wrote: Based on https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code#C.2B.2B_and_Mozilla_standard_libraries (thanks njn!) we can use std::unordered_map and std::unordered_set at least. That still won't give us a convenient to use queue/deque through. Do you not see the giant note at the top of that page saying Don't trust this information? As the person who wrote 95% of that page, I can tell you that the STL section does not contain any information about what can and can't be used in Mozilla code, largely as a result of trying to balance the need to explain what can be used in Mozilla versus the desire to list the minimum features in C++11 code. This is particularly true for the ADT section, as the executive decision has been repeatedly affirmed to avoid STL containers as much as possible in Mozilla code. Note, for example, that STLport (the version we have in the tree) provides std::tr1::unordered_map, not std::unordered_map. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should we disable autoplay feature of HTMLMediaElement on mobile?
On 12/11/2013 3:07 PM, Tetsuharu OHZEKI wrote: 2013/12/11 Hubert Figuière h...@mozilla.com: It should be noted that here on desktop I do have `media.autoplay.enabled=false` (I hate videos that autoplay) and this still doesn't stop YouTube. [1] I seem that Gecko's implementation of HTMLMediaElement.play() doesn't handle `media.autoplay.enabled`. http://mxr.mozilla.org/mozilla-central/source/content/html/content/src/HTMLMediaElement.cpp#2112 Because play() and autoplay are different. We could add a pref that caused HTMLMediaElement.play() to only work if called from inside a user generated event handler (i.e. mouse click, key press). That would achieve most of what you want, and is very easy to implement. Chris P. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should we disable autoplay feature of HTMLMediaElement on mobile?
On Dec 10, 2013 6:30 PM, Chris Pearce cpea...@mozilla.com wrote: We could add a pref that caused HTMLMediaElement.play() to only work if called from inside a user generated event handler (i.e. mouse click, key press). That would achieve most of what you want, and is very easy to implement. This seems like it might break e.g. games. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should we disable autoplay feature of HTMLMediaElement on mobile?
On 12/11/2013 4:18 PM, Alex Jordan wrote: On Dec 10, 2013 6:30 PM, Chris Pearce cpea...@mozilla.com wrote: We could add a pref that caused HTMLMediaElement.play() to only work if called from inside a user generated event handler (i.e. mouse click, key press). That would achieve most of what you want, and is very easy to implement. This seems like it might break e.g. games. This would be preffed *off* by default of course. We could make this only for video elements in documents. But that's still defeatable. You really want something like what roc suggested. This is just easier. Chris P. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
