Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
Note that we have bug 990804 opened for providing such a service. Unfortunately, this bug is currently stalled because we apparently didn't have a clear idea of what API clients would need. I'm willing to pick this for Q2, if there is interest. Cheers, David On 26/03/15 07:46, Randell Jesup wrote: As a result of some conversations on IRC, it came to light that a bunch of blocking IO operations were apparently moved off of MainThread (presumably to avoid UI 'jank'), and instead Dispatched to STREAMTRANSPORTSERVICE (aka STS thread). Some examples pointed out to me: FilePicker, the spell-checker, the DeviceStorage DOM code, DOM cache code in Manager.cpp (via BodyStartWriteStream()), even perhaps ResolvedCallback in ServiceWorkers. (I haven't looked closely at all of the uses yet.) Good, right? No jank in the UI! No - now the jank is in the network code! STS is the primary network thread, and blocking that without a Darn Good Reason will cause all sorts of negative effects. Page loads, XHR, WebSockets, WebRTC (which will react by adding delay to cover the 'jitter' in apparent network RTT) and many other things will have small-to-large negative impacts. Perhaps some of these listed above (and others) don't block or even have a legitimate need to run on STS thread - ok, if so, comment as to why it's ok/needed. Everyone else should not be using STS thread as a convenient target It does point out that with the need to get a lot of blocking operations off of MainThread, it would help other modules to have a single service or ThreadPool for dumping such operations to. This would mean less code, less incorrect/undone thread shutdowns/etc. Thoughts? (Perhaps such a service could use LazyIdleThreads, which will shut themselves down after inactivity.) -- David Rajchenbach-Teller, PhD Performance Team, Mozilla signature.asc Description: OpenPGP digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
libxul liscensing
Hi All, I've been googling now for a far too long trying to find definitive documentation about the license governing the usage and distribution of libxul. All I can find is a very short wiki page that states that the licensing of xulrunner is being discussed internally to address some challenges, with a hint about gpl/lgpl code being mixed in somewhere. It's not very clear if this is about xulrunner the executable or the contents of the xulrunner SDK. Maybe my google-fu has gotten a little weak or perhaps I'm just missing something, so I signed up to ask you nice folks if you could help point me to some documentation that would make this clearer for me. I suppose while I'm at it, I'll pose my second question that I also can't find answers to, which is, is it possible to build libxul into a series of smaller shared libraries instead of a monolithic 30+ meg dll? Thanks for taking the time to even read my questions, and thanks in advance for any info you can provide me. :) Regards, -- Jesse Nicholson ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
On Thu, Mar 26, 2015 at 2:46 AM, Randell Jesup rjesup.n...@jesup.org wrote: Some examples pointed out to me: FilePicker, the spell-checker, the DeviceStorage DOM code, DOM cache code in Manager.cpp (via BodyStartWriteStream()), even perhaps ResolvedCallback in ServiceWorkers. (I haven't looked closely at all of the uses yet.) Sorry for this. Obviously there has been some confusion as I was explicitly directed towards STS during the DOM Cache development. (I even thought there was a separate SocketTransportService which was different from StreamTransportService.) In any case, I wrote a bug to fix the Cache issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1147850 I will try to fix this in the next couple weeks. Sorry again. Ben ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: libxul liscensing
On Thu, Mar 26, 2015 at 6:26 AM, Jesse Nicholson ascensionsyst...@gmail.com wrote: Hi All, I've been googling now for a far too long trying to find definitive documentation about the license governing the usage and distribution of libxul. All I can find is a very short wiki page that states that the licensing of xulrunner is being discussed internally to address some challenges, with a hint about gpl/lgpl code being mixed in somewhere. It's not very clear if this is about xulrunner the executable or the contents of the xulrunner SDK. Maybe my google-fu has gotten a little weak or perhaps I'm just missing something, so I signed up to ask you nice folks if you could help point me to some documentation that would make this clearer for me. I suppose while I'm at it, I'll pose my second question that I also can't find answers to, which is, is it possible to build libxul into a series of smaller shared libraries instead of a monolithic 30+ meg dll? Thanks for taking the time to even read my questions, and thanks in advance for any info you can provide me. :) Regards, -- Jesse Nicholson ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform libxul binaries made by Mozilla are made available under the terms of the MPL 2. https://www.mozilla.org/MPL/ - Kyle ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: [JS-internals] SpiderMonkey and XPConnect style changing from |T *p| to |T* p|
On Fri, Mar 27, 2015 at 10:25 AM, Robert O'Callahan rob...@ocallahan.org wrote: Switching method names to start with a capital letter would be helpful too ... especially where it's leaked into MFBT. FWIW, here is the contents of mfbt/STYLE: MFBT uses standard Mozilla style, with the following exceptions. - Some of the files use a lower-case letter at the start of function names. This is because MFBT used to use a different style, and was later converted to standard Mozilla style. These functions have not been changed to use an upper-case letter because it would cause a lot of churn in other parts of the codebase. However, new files should follow standard Mozilla style and use an upper-case letter at the start of function names. - Imported third-party code (such as decimal/*, double-conversion/*, and lz4*) remains in its original style. Getting rid of that first exception would be a fine thing, IMO. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: SpiderMonkey and XPConnect style changing from |T *p| to |T* p|
On 27 March 2015 at 01:19, Bobby Holley bobbyhol...@gmail.com wrote: Can we switch from 4-space to 2-space indentation at some point too? Together, those would remove the most glaring formatting differences. The others (bracing single-line consequents, aNamingWarts, etc) might be harder. I've done all of that and more on a few directories (e.g. xpcom/ in bug 1046841). If we reach consensus regarding js/ style, I could take care of the conversion over the summer. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: FYI: Serious windows bug affecting WebRTC when mem use is over 2GB
I guess we can add a command line option to our executable that calls the function and prints the results and exits and then invoke ourselves to do this in a new process and parse the output. What a silly bug. Thanks, Andreas Sent from Mobile. On Mar 26, 2015, at 07:03, Daniel Stenberg dan...@haxx.se wrote: On Thu, 26 Mar 2015, Benjamin Smedberg wrote: What is the largest buffer that we can expect to need? Since VM allocation happens in 64k boundaries, is it sufficient to just use a 64k buffer for this? As per a recent comment in the bug however, it doesn't work to just reserve some memory in the lower region: https://bugzilla.mozilla.org/show_bug.cgi?id=1107702#c20 -- / daniel.haxx.se ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Customize what's compiled into libXUL
I'm wondering if anyone can point me to some documentation on customizing what get's built into libxul. All I can find is outdated random forum posts about changing defines in a nsStaticXULComponents, specifically XUL_MODULES define, but that information seems to no longer apply. I'm looking to create a stripped down version that only includes the functionality from the layout module (for dom parsing, css selectors, so on). The link in the documentation that points to the appropriate mailing list is broken/outdated, so hopefully I've written to the right list. Thanks in advance. -- -JN ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
dom/media has a very useful system called SharedThreadPool that gives you static on-demand access to a named (and refcounted) threadpool. Probably just moving that to XPCOM and then coming up with a naming convention (MiscThreadPool or something) might solve this problem. On Thu, Mar 26, 2015 at 8:37 AM, Randell Jesup rjesup.n...@jesup.org wrote: Can we stop exposing the socket transport service's nsIEventTarget outside of Necko? If we move media/mtransport to necko... or make an exception for it (and dom/network/UDPSocket and TCPSocket, etc). Things that remove loaded footguns (or at least lock them down) are good. Glad the major real problem was too-similar-names (I'd never heard of STREAMTRANSPORTSERVICE (or if I had, it had been long-forgotten, or mis-read as SOCKETTRANSPORTSERVICE)). -- Randell Jesup, Mozilla Corp remove news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: SpiderMonkey and XPConnect style changing from |T *p| to |T* p|
Is the bug tasking this marked dev-doc-needed? We will have SD A bunch of cleanup to do On Thursday, March 26, 2015, Jan De Mooij jdemo...@mozilla.com wrote: Hi all, After some discussion, we want to switch SpiderMonkey and XPConnect style for pointers/references from |T *p| to |T* p| (and |T ref| to |T ref|). This matches the rest of the project and will make life easier for developers working on multiple parts of the tree. Although this will break a lot of patches, you can run the script in bug 1144366 [0] on patch files too and they should apply cleanly after that (make sure you have backups): restyle.py --files patch1 [patch2..] The plan is to land this change next weekend or early next week. We'll likely land this on aurora/beta/ESR as well, to avoid painful uplifts. Finally, a common argument for the current style is that it might be clearer when you declare multiple pointers, like this: int *x, *y; // x and y are int* However, I think this is pretty confusing either way, and both our style guide [1] and Stroustrup [2] suggest sticking to one pointer per declaration. I'll write a separate patch to do this. I'm very sorry for the rebase and merge pain, but we think this is worth doing. Thanks, Jan [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1144366 [1] https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style#Declarations [2] http://www.stroustrup.com/bs_faq2.html#whitespace ___ dev-platform mailing list dev-platform@lists.mozilla.org javascript:; https://lists.mozilla.org/listinfo/dev-platform -- Eric Shepherd Senior Technical Writer Mozilla Blog: http://www.bitstampede.com/ Twitter: http://twitter.com/sheppy ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Non-UTF-8 file paths on Gtk platforms
==Summary== I suggest that we either decide not to support non-UTF-8 file paths in Gecko/Firefox on Gtk platforms or, failing that, use the glib facilities to convert between Unicode and file paths. Opinions? ==Relevant bugs== https://bugzilla.mozilla.org/show_bug.cgi?id=960957 https://bugzilla.mozilla.org/show_bug.cgi?id=978056 ==Details== On (non-OS X) *nix, file paths are byte strings without file system or kernel-level semantics for the bytes with the most significant bit set. While on *nix nsIFile holds a byte strings as the path, JS strings are UTF-16, so when paths are exposed to JS code, there's a need to convert between UTF-16 and byte strings, which requires deciding what encoding to use for the byte strings. On OS X, Gonk and Android, we assume that the encoding of the byte strings is always UTF-8. However, on Gtk platforms, except when using the OS.File API, we try to determine the encoding by calling nl_langinfo(CODESET) or by falling back onto ISO-8859-1. Most of the time, the result is UTF-8 anyway, since UTF-8 has been the default of Linux distros for years. Red Hat 8, released in 2002, defaulted to UTF-8. On the Debian side, UTF-8 has been the default since 2007. The most likely deviation from the rule is launching Gecko with the POSIX C locale, which results in the file path encoding being US-ASCII--i.e. non-ASCII paths fail. The OS.File API assumes UTF-8 even on Gtk platform even when nl_langinfo(CODESET) says something other than UTF-8. The above-described behavior differs from how glib/Gtk apps are supposed to behave. The way glib/Gtk apps are supposed to behave is leaving conversion between Unicode and the file path encoding to glib, which behaves as follows: 1) If the G_FILENAME_ENCODING environment variable is set, treat it as a comma-separated list and assume the first item is the file name encoding. (A special token @locale means taking the encoding from the locale.) 2) Otherwise, if the G_BROKEN_FILENAMES environment variable is set, take use the encoding from the locale as the file name encoding. 3) Otherwise, use UTF-8 as the file name encoding. ==Suggestion 1== Since we've managed to ship with OS.File failing if the file path encoding is not UTF-8 and since UTF-8 has been the default since 2002 on the Red Hat side and since 2007 on the Debian side, let's drop support for non-UTF-8 file name paths on all *nix platforms. ==Suggestion 2== If Suggestion 1 is considered too radical and we want to keep supporting configurations that use non-UTF-8 file paths because the systems, let's at least drop some legacy code on our side and behave like a normal Gtk app: 1) Upon initialization, make nsNativeCharsetUtils convert a UTF-8 string using g_filename_to_utf8(). If the resulting string has the same bytes, set gIsNativeUTF8 to true. 2) If gIsNativeUTF8 is true, implement NS_CopyNativeToUnicode as CopyUTF8toUTF16 and implement NS_CopyUnicodeToNative as CopyUTF16toUTF8. 3) If nsNativeCharsetConverter::gIsNativeUTF8 is false, implement NS_CopyNativeToUnicode as g_filename_to_utf8() followed by CopyUTF8toUTF16 and implement NS_CopyUnicodeToNative as CopyUTF16toUTF8 followed by g_filename_from_utf8(). (This results in a UTF-8 intermediate copy when the file system, but only in the unusual case of the file system encoding not being UTF-8. Also, we get rid of a bunch of old code and unify our behavior with other Gtk apps.) 4) Make OS.File use NS_CopyNativeToUnicode and NS_CopyUnicodeToNative for converting between JS strings and file system path byte strings. -- Henri Sivonen hsivo...@hsivonen.fi https://hsivonen.fi/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: FYI: Serious windows bug affecting WebRTC when mem use is over 2GB
On 3/26/2015 3:00 AM, Randell Jesup wrote: Force a buffer in 2GB memory and always copy into/out of that buffer? That may work, other than it's inconvenient to force a buffer into 2GB space at the time when you need it, and thus we'd have to perma-allocate a large enough buffer for this. You don't have to commit it, you only have to reserve the address space. (Note GetAdapters*() is typically used with a first call to get the size, then a second to get the real data, or with a 16K-ish buffer then redone if that isn't enough.) What is the largest buffer that we can expect to need? Since VM allocation happens in 64k boundaries, is it sufficient to just use a 64k buffer for this? What thread(s) are these functions called on? Can we serialize access to them so that we can allocate just one buffer at startup and use that singleton for all the callers? --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
Actually, I'm going to steal bug 990804 and see if we can get something worked out now. My plan is just to duplicate the STS code with a different XPCOM uuid for now. On Thu, Mar 26, 2015 at 9:29 AM, Benjamin Kelly bke...@mozilla.com wrote: On Thu, Mar 26, 2015 at 2:46 AM, Randell Jesup rjesup.n...@jesup.org wrote: Some examples pointed out to me: FilePicker, the spell-checker, the DeviceStorage DOM code, DOM cache code in Manager.cpp (via BodyStartWriteStream()), even perhaps ResolvedCallback in ServiceWorkers. (I haven't looked closely at all of the uses yet.) Sorry for this. Obviously there has been some confusion as I was explicitly directed towards STS during the DOM Cache development. (I even thought there was a separate SocketTransportService which was different from StreamTransportService.) In any case, I wrote a bug to fix the Cache issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1147850 I will try to fix this in the next couple weeks. Sorry again. Ben ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: FYI: Serious windows bug affecting WebRTC when mem use is over 2GB
On Thu, 26 Mar 2015, Benjamin Smedberg wrote: What is the largest buffer that we can expect to need? Since VM allocation happens in 64k boundaries, is it sufficient to just use a 64k buffer for this? As per a recent comment in the bug however, it doesn't work to just reserve some memory in the lower region: https://bugzilla.mozilla.org/show_bug.cgi?id=1107702#c20 -- / daniel.haxx.se ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: FYI: Serious windows bug affecting WebRTC when mem use is over 2GB
On Fri, Mar 27, 2015 at 5:05 AM, Andreas Gal andreas@gmail.com wrote: I guess we can add a command line option to our executable that calls the function and prints the results and exits and then invoke ourselves to do this in a new process and parse the output. What a silly bug. Surely we can ship a little executable so we don't have to CreateProcess plugin-container. What a crazy bug. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Customize what's compiled into libXUL
On 2015-03-26 3:07 PM, Jesse Nicholson wrote: I'm wondering if anyone can point me to some documentation on customizing what get's built into libxul. All I can find is outdated random forum posts about changing defines in a nsStaticXULComponents, specifically XUL_MODULES define, but that information seems to no longer apply. I'm looking to create a stripped down version that only includes the functionality from the layout module (for dom parsing, css selectors, so on). The link in the documentation that points to the appropriate mailing list is broken/outdated, so hopefully I've written to the right list. That's not a supported configuration, and chances are that you would need to modify Gecko in order to make it build and function in such a mode. Depending on how much time you're willing to spend on this, it may not be practical, but that's hard to say without knowing exactly what components you would want to disable. I really suggest against doing this if at all possible. But in order to answer your question, I don't think we have any updated docs. You can look in configure.in in the root of the tree for options on what things can be disabled, but don't be surprised if using some of them gives you broken builds, etc. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
SpiderMonkey and XPConnect style changing from |T *p| to |T* p|
Hi all, After some discussion, we want to switch SpiderMonkey and XPConnect style for pointers/references from |T *p| to |T* p| (and |T ref| to |T ref|). This matches the rest of the project and will make life easier for developers working on multiple parts of the tree. Although this will break a lot of patches, you can run the script in bug 1144366 [0] on patch files too and they should apply cleanly after that (make sure you have backups): restyle.py --files patch1 [patch2..] The plan is to land this change next weekend or early next week. We'll likely land this on aurora/beta/ESR as well, to avoid painful uplifts. Finally, a common argument for the current style is that it might be clearer when you declare multiple pointers, like this: int *x, *y; // x and y are int* However, I think this is pretty confusing either way, and both our style guide [1] and Stroustrup [2] suggest sticking to one pointer per declaration. I'll write a separate patch to do this. I'm very sorry for the rebase and merge pain, but we think this is worth doing. Thanks, Jan [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1144366 [1] https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style#Declarations [2] http://www.stroustrup.com/bs_faq2.html#whitespace ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: SpiderMonkey and XPConnect style changing from |T *p| to |T* p|
\o/ Can we switch from 4-space to 2-space indentation at some point too? Together, those would remove the most glaring formatting differences. The others (bracing single-line consequents, aNamingWarts, etc) might be harder. On Thu, Mar 26, 2015 at 3:52 PM, Jan De Mooij jdemo...@mozilla.com wrote: Hi all, After some discussion, we want to switch SpiderMonkey and XPConnect style for pointers/references from |T *p| to |T* p| (and |T ref| to |T ref|). This matches the rest of the project and will make life easier for developers working on multiple parts of the tree. Although this will break a lot of patches, you can run the script in bug 1144366 [0] on patch files too and they should apply cleanly after that (make sure you have backups): restyle.py --files patch1 [patch2..] The plan is to land this change next weekend or early next week. We'll likely land this on aurora/beta/ESR as well, to avoid painful uplifts. Finally, a common argument for the current style is that it might be clearer when you declare multiple pointers, like this: int *x, *y; // x and y are int* However, I think this is pretty confusing either way, and both our style guide [1] and Stroustrup [2] suggest sticking to one pointer per declaration. I'll write a separate patch to do this. I'm very sorry for the rebase and merge pain, but we think this is worth doing. Thanks, Jan [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1144366 [1] https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style#Declarations [2] http://www.stroustrup.com/bs_faq2.html#whitespace ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: SpiderMonkey and XPConnect style changing from |T *p| to |T* p|
On Fri, Mar 27, 2015 at 12:19 PM, Bobby Holley bobbyhol...@gmail.com wrote: \o/ Can we switch from 4-space to 2-space indentation at some point too? Together, those would remove the most glaring formatting differences. The others (bracing single-line consequents, aNamingWarts, etc) might be harder. Switching method names to start with a capital letter would be helpful too ... especially where it's leaked into MFBT. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
dom/network/UDPSocket doesn't use SocketTransportService directly so it doesn't create exception. It should be ok to leave it outside of Necko. Best Regards, Shih-Chiang Chien Mozilla Taiwan On Thu, Mar 26, 2015 at 11:37 PM, Randell Jesup rjesup.n...@jesup.org wrote: Can we stop exposing the socket transport service's nsIEventTarget outside of Necko? If we move media/mtransport to necko... or make an exception for it (and dom/network/UDPSocket and TCPSocket, etc). Things that remove loaded footguns (or at least lock them down) are good. Glad the major real problem was too-similar-names (I'd never heard of STREAMTRANSPORTSERVICE (or if I had, it had been long-forgotten, or mis-read as SOCKETTRANSPORTSERVICE)). -- Randell Jesup, Mozilla Corp remove news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
FYI: Serious windows bug affecting WebRTC when mem use is over 2GB
Thanks to detective work by a subscriber to dev.media (Tor-Einar Jarnbjo), we've found the cause of unexplained ICE (NAT-traversal) failures in WebRTC on Windows (bug 1107702). This may also affect the code in netwerk that tracks the network link status. It turns out that 32bit Windows programs with the large-address-aware flag (to use 2GB of memory) will fail calls to GetHostByName/GetHostByAddress/GetAdaptersInfo/GetAdaptersAddresses *if* you're currently using 2GB of process memory (likely it's if the buffer you pass in is above the 2GB point). This bug (http://support.microsoft.com/en-us/kb/2588507) affects Vista/Server2008/Win7, and unfortunately seems to only be fixable by a user-installation of a hotfix. :-( We're pretty sure it's the cause of some previously-unexplained ICE failures in WebRTC/Hello. Since switching all Windows64 Firefox installs to 64-bit isn't on a short-term roadmap, and there is no reasonable technical workaround at the code level we know of, our only other options are to drop 2GB support in Windows (ouch) or lean heavily on Microsoft to ship the fix to all Windows users affected in their automatic updates. The only other idea I can think of would be to proxy all such uses to an entire separate *process*. I can't tell you how sick it makes me to suggest that... Anyone have a better idea? Or does anyone want to drop above-2GB support? -- Randell Jesup, Mozilla Corp remove news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
PSA: Network 'jank' - get your blocking IO off of STS thread!
As a result of some conversations on IRC, it came to light that a bunch of blocking IO operations were apparently moved off of MainThread (presumably to avoid UI 'jank'), and instead Dispatched to STREAMTRANSPORTSERVICE (aka STS thread). Some examples pointed out to me: FilePicker, the spell-checker, the DeviceStorage DOM code, DOM cache code in Manager.cpp (via BodyStartWriteStream()), even perhaps ResolvedCallback in ServiceWorkers. (I haven't looked closely at all of the uses yet.) Good, right? No jank in the UI! No - now the jank is in the network code! STS is the primary network thread, and blocking that without a Darn Good Reason will cause all sorts of negative effects. Page loads, XHR, WebSockets, WebRTC (which will react by adding delay to cover the 'jitter' in apparent network RTT) and many other things will have small-to-large negative impacts. Perhaps some of these listed above (and others) don't block or even have a legitimate need to run on STS thread - ok, if so, comment as to why it's ok/needed. Everyone else should not be using STS thread as a convenient target It does point out that with the need to get a lot of blocking operations off of MainThread, it would help other modules to have a single service or ThreadPool for dumping such operations to. This would mean less code, less incorrect/undone thread shutdowns/etc. Thoughts? (Perhaps such a service could use LazyIdleThreads, which will shut themselves down after inactivity.) -- Randell Jesup, Mozilla Corp remove news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: FYI: Serious windows bug affecting WebRTC when mem use is over 2GB
Force a buffer in 2GB memory and always copy into/out of that buffer? That may work, other than it's inconvenient to force a buffer into 2GB space at the time when you need it, and thus we'd have to perma-allocate a large enough buffer for this. (Note GetAdapters*() is typically used with a first call to get the size, then a second to get the real data, or with a 16K-ish buffer then redone if that isn't enough.) Each user of GetAdapters*() (I don't think we have any uses of GetHostByName/Address) would need to pre-allocate a buffer during startup probably (or use a static buffer). You still would have to dynamically allocate one if the static buffer wasn't big enough, and that could then fail of course. Better than turning off 2GB memory, though. -- Randell Jesup, Mozilla Corp remove news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
On 2015-03-26 11:14 AM, Patrick McManus wrote: good catch.. looking at https://mxr.mozilla.org/mozilla-central/ident?i=NS_SOCKETTRANSPORTSERVICE_CONTRACTID the only uses I see other than the one Ehsan unearthed are expected.. so maybe that's the sum of the short term work. Great! Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1147913 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
Nathan suggests the following plan to avoid code duplication: 1) Move STS code into xpcom/io under a different name and XPCOM uuid. 2) Make the necko STS a no-op inherit from the xpcom code with a different uuid. This would give us two service instances without duplicating any code. Also, I'll clarify this comment to indicate what the different sservice instances are for. Right now it kind of suggests people use STS for file streams: https://dxr.mozilla.org/mozilla-central/source/netwerk/base/nsIStreamTransportService.idl#14 Any objections? Thanks. Ben On Thu, Mar 26, 2015 at 10:08 AM, Patrick McManus mcma...@ducksong.com wrote: thanks bkelly On Thu, Mar 26, 2015 at 9:01 AM, Benjamin Kelly bke...@mozilla.com wrote: Actually, I'm going to steal bug 990804 and see if we can get something worked out now. My plan is just to duplicate the STS code with a different XPCOM uuid for now. On Thu, Mar 26, 2015 at 9:29 AM, Benjamin Kelly bke...@mozilla.com wrote: On Thu, Mar 26, 2015 at 2:46 AM, Randell Jesup rjesup.n...@jesup.org wrote: Some examples pointed out to me: FilePicker, the spell-checker, the DeviceStorage DOM code, DOM cache code in Manager.cpp (via BodyStartWriteStream()), even perhaps ResolvedCallback in ServiceWorkers. (I haven't looked closely at all of the uses yet.) Sorry for this. Obviously there has been some confusion as I was explicitly directed towards STS during the DOM Cache development. (I even thought there was a separate SocketTransportService which was different from StreamTransportService.) In any case, I wrote a bug to fix the Cache issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1147850 I will try to fix this in the next couple weeks. Sorry again. Ben ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
media uses it by agreement and in an appropriate way to support rtcweb. On Thu, Mar 26, 2015 at 10:20 AM, Kyle Huey m...@kylehuey.com wrote: Can we stop exposing the socket transport service's nsIEventTarget outside of Necko? - Kyle On Thu, Mar 26, 2015 at 8:14 AM, Patrick McManus mcma...@ducksong.com wrote: good catch.. looking at https://mxr.mozilla.org/mozilla-central/ident?i=NS_SOCKETTRANSPORTSERVICE_CONTRACTID the only uses I see other than the one Ehsan unearthed are expected.. so maybe that's the sum of the short term work. On Thu, Mar 26, 2015 at 10:06 AM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: On 2015-03-26 11:00 AM, Kyle Huey wrote: On Thu, Mar 26, 2015 at 7:49 AM, Patrick McManus mcma...@ducksong.com wrote: Is this thread mostly just confusion from these things sounding so much alike? Or am I confused now? Most likely. Does anyone have actual data to show that this is a problem? There's some truth to it. Looks like some code uses the *socket* transport service when it probably means *stream* transport service. Example: http://mxr.mozilla.org/mozilla-central/source/dom/ workers/ServiceWorkerEvents.cpp#249 But other examples such as DOM Cache are not affected as far as I can tell. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
: On Thu, Mar 26, 2015 at 2:46 AM, Randell Jesup rjesup.n...@jesup.org wrote: t. (I even thought there was a separate SocketTransportService which was different from StreamTransportService.) You're right they are different things. The socket transport service is a single thread that does most of the low level networking - https://mxr.mozilla.org/mozilla-central/source/netwerk/base/nsSocketTransportService2.cpp#487 .. blocking this thread would be very bad. and the stream transport service is a thread pool that is used for buffering management primarily, etc.. https://mxr.mozilla.org/mozilla-central/source/netwerk/base/nsStreamTransportService.cpp#485 .. I'm not sure how I feel about overloading arbitrary other functionality here, but its certainly less damaging than blocking the single socket thread. Is this thread mostly just confusion from these things sounding so much alike? Or am I confused now? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
Can we stop exposing the socket transport service's nsIEventTarget outside of Necko? - Kyle On Thu, Mar 26, 2015 at 8:14 AM, Patrick McManus mcma...@ducksong.com wrote: good catch.. looking at https://mxr.mozilla.org/mozilla-central/ident?i=NS_SOCKETTRANSPORTSERVICE_CONTRACTID the only uses I see other than the one Ehsan unearthed are expected.. so maybe that's the sum of the short term work. On Thu, Mar 26, 2015 at 10:06 AM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: On 2015-03-26 11:00 AM, Kyle Huey wrote: On Thu, Mar 26, 2015 at 7:49 AM, Patrick McManus mcma...@ducksong.com wrote: Is this thread mostly just confusion from these things sounding so much alike? Or am I confused now? Most likely. Does anyone have actual data to show that this is a problem? There's some truth to it. Looks like some code uses the *socket* transport service when it probably means *stream* transport service. Example: http://mxr.mozilla.org/mozilla-central/source/dom/ workers/ServiceWorkerEvents.cpp#249 But other examples such as DOM Cache are not affected as far as I can tell. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
thanks bkelly On Thu, Mar 26, 2015 at 9:01 AM, Benjamin Kelly bke...@mozilla.com wrote: Actually, I'm going to steal bug 990804 and see if we can get something worked out now. My plan is just to duplicate the STS code with a different XPCOM uuid for now. On Thu, Mar 26, 2015 at 9:29 AM, Benjamin Kelly bke...@mozilla.com wrote: On Thu, Mar 26, 2015 at 2:46 AM, Randell Jesup rjesup.n...@jesup.org wrote: Some examples pointed out to me: FilePicker, the spell-checker, the DeviceStorage DOM code, DOM cache code in Manager.cpp (via BodyStartWriteStream()), even perhaps ResolvedCallback in ServiceWorkers. (I haven't looked closely at all of the uses yet.) Sorry for this. Obviously there has been some confusion as I was explicitly directed towards STS during the DOM Cache development. (I even thought there was a separate SocketTransportService which was different from StreamTransportService.) In any case, I wrote a bug to fix the Cache issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1147850 I will try to fix this in the next couple weeks. Sorry again. Ben ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
On 2015-03-26 11:00 AM, Kyle Huey wrote: On Thu, Mar 26, 2015 at 7:49 AM, Patrick McManus mcma...@ducksong.com wrote: Is this thread mostly just confusion from these things sounding so much alike? Or am I confused now? Most likely. Does anyone have actual data to show that this is a problem? There's some truth to it. Looks like some code uses the *socket* transport service when it probably means *stream* transport service. Example: http://mxr.mozilla.org/mozilla-central/source/dom/workers/ServiceWorkerEvents.cpp#249 But other examples such as DOM Cache are not affected as far as I can tell. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
Can we stop exposing the socket transport service's nsIEventTarget outside of Necko? If we move media/mtransport to necko... or make an exception for it (and dom/network/UDPSocket and TCPSocket, etc). Things that remove loaded footguns (or at least lock them down) are good. Glad the major real problem was too-similar-names (I'd never heard of STREAMTRANSPORTSERVICE (or if I had, it had been long-forgotten, or mis-read as SOCKETTRANSPORTSERVICE)). -- Randell Jesup, Mozilla Corp remove news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
good catch.. looking at https://mxr.mozilla.org/mozilla-central/ident?i=NS_SOCKETTRANSPORTSERVICE_CONTRACTID the only uses I see other than the one Ehsan unearthed are expected.. so maybe that's the sum of the short term work. On Thu, Mar 26, 2015 at 10:06 AM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: On 2015-03-26 11:00 AM, Kyle Huey wrote: On Thu, Mar 26, 2015 at 7:49 AM, Patrick McManus mcma...@ducksong.com wrote: Is this thread mostly just confusion from these things sounding so much alike? Or am I confused now? Most likely. Does anyone have actual data to show that this is a problem? There's some truth to it. Looks like some code uses the *socket* transport service when it probably means *stream* transport service. Example: http://mxr.mozilla.org/mozilla-central/source/dom/ workers/ServiceWorkerEvents.cpp#249 But other examples such as DOM Cache are not affected as far as I can tell. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Network 'jank' - get your blocking IO off of STS thread!
One more place: https://bugzilla.mozilla.org/show_bug.cgi?id=1147921 -hb- On 3/26/2015 16:14, Patrick McManus wrote: good catch.. looking at https://mxr.mozilla.org/mozilla-central/ident?i=NS_SOCKETTRANSPORTSERVICE_CONTRACTID the only uses I see other than the one Ehsan unearthed are expected.. so maybe that's the sum of the short term work. On Thu, Mar 26, 2015 at 10:06 AM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: On 2015-03-26 11:00 AM, Kyle Huey wrote: On Thu, Mar 26, 2015 at 7:49 AM, Patrick McManus mcma...@ducksong.com wrote: Is this thread mostly just confusion from these things sounding so much alike? Or am I confused now? Most likely. Does anyone have actual data to show that this is a problem? There's some truth to it. Looks like some code uses the *socket* transport service when it probably means *stream* transport service. Example: http://mxr.mozilla.org/mozilla-central/source/dom/ workers/ServiceWorkerEvents.cpp#249 But other examples such as DOM Cache are not affected as far as I can tell. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
