Re: [webkit-gtk] XmlHttprequest? leaking ?
Nope. Create a GTK tab with a webview and close it and you lost 3MB.
I have been working on trying to narrow down where the leaks are.
Either in webkit or in the GTK parts.
Memory leaks are the bane of webkit gtk at this point.
On 06/12/13 03:31, Vellemans, Noel wrote:
Hi,
Still struggling with leaks into XMLHTTP-request.
OBSERVATIONS :
The VmSize of the Web-browser process (GtkLauncher on ARM anx x64) at
startup is typically VmSize:=105000 kB, after a 4 XMLhttp-requests
the memory size is increased to VmSize:=387448 kB. ( and keeps on
increasing)
When doing more xml-http-requests the browser finally dies due to out of
memory.
Anyone that is having similar issue's ?
Any clue where to look (start looking at) ?
Anyone out that has ever tried to do XMLHTTP requests in a page that is
refreshing every second ?
I've been stepping/building.. etc.. for multiple weeks now, I'm I the
only one with this issue ?
Regards,
Noel
Message: 1
Date: Thu, 23 May 2013 10:33:51 +0200
From: "Vellemans, Noel"
To:
Subject: [webkit-gtk] XmlHttprequest? leak
Message-ID: <1531E53627F1F749B4FE809BF2A4EB6704295E59@wetmex10>
Content-Type: text/plain; charset="US-ASCII"
Hi all,
I'm using (building) WekKitGtk+ (1.10.2) { On Arm-9 and on PC-x64 }
Both seem to suffer from the same problem. Leaking XMLhttpRequest.
( ive tried some simple html/javascript that is doing XMLhttpRequest in
a repetitive way, )
It seems that "repetitive" XMLhttpRequest's are leaking LOTS of memory.
{ for example ("GET", "/sometxt.txt?" + numRep, true); } Anyone that
can tell me where to look at (I'm stuck for a couple of weeks now).
Anyone that can give me a sample that is NOT leaking ?
I even included http://trac.webkit.org/changeset/143619. (on
recommendation of Martin Robinson) .
Same result leaking XMLhttp-request (until browsers dies).
OBSERVATIONS:
The VmSize of the Web-browser process (GtkLauncher on ARM-9) at startup
is typically VmSize:=105000 kB, after a 4 XMLhttp-requests the
memory size is increased to VmSize:=387448 kB. ( and keeps on
increasing)
When doing more xml-http-requests the browser finally dies due to out of
memory.
NOTE: when not doing the xml-httprequests but some onther javascripting
(that increments a counter on the same time-interval) the Browser
(GtkLauncher on ARM-9) VmSize 'ceils' at 205000 kB (aprox), and I can
let the browser run for days, [ have one unit-running for 4 weeks (as
test)] without crashing .
Regards Noel
<< JSC-code - snap-shot >>
function Global_onreadystatechange()
{
numReady++;
if(this.readyState == 4 && this.status == 200)
{
this.onreadystatechange = null;
numResp++;
}
}
function Global_Request()
{
if(xhr)
{
// the ? numRep should make it harder for cache
xhr.open("GET", "/sometxt.txt?" + numRep, true);
xhr.setRequestHeader("Content-Type","text/plain");
xhr.onreadystatechange = Global_onreadystatechange;
xhr.send(null);
}
}
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-gtk
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-gtk
Re: [webkit-gtk] Progress about TLS handling in WebKit2
On Thu, Jul 05, 2012 at 12:23:11PM +0200, Carlos Garcia Campos wrote: > After thinking more about it and looking at ephy and other browsers > code, I think that we want to support at least two features regarding > the TLS handling: > > - Provide certificate information to apps, so that they can set the > security level and provide information about the certificate, like > chromium does in the location entry. > > - Allow apps to decide whether accept or deny a website load depending > on the certificate. This would be fantastic! I'd love to see this go in instead of jumping through the hoops we jump through now. > > Provide certificate information to apps > - > > In WebKit2 there's already a PlatformCertificateInfo abstraction that is > sent to the UI process as part of the loadCommited message and stored in > the WebFrameProxy. The C API allows to get the certificate info from a > WKFrame. This is always sent, no matter if the are tls errors or not. > > In our case, we could implement the PlatformCertificateInfo based on > GTlsCertificate object and serialize/deserialze it using the > "certificate" property. We should probably include the TLS errors too. > > Current epiphany code, gets the soup message flags from the main > resource request when load status is committed and simply checks whether > SOUP_MESSAGE_CERTIFICATE_TRUSTED is present or not to set the security > level as HIGH or BROKEN. SoupMessage sets that flag depending on whether > tls-errors is empty or not, so including tls-errors as part of the > PlatformCertificateInfo serialization would allow us to decide whether > the certificate can be trusted or not (and it provides more information, > for example, we could make the lock icon clickable and give more details > about the errors in a dialog or in a tooltip). > > So, the question is how to expose this in the API. First, we want to > provide both, the certificate info and the tls-errors flags, so we could > give both as parameters of a signal, or we could add a simple wrapper > WebKitCertificateInfo or WebKitCertificate containing both. I think we > have several options: > > a) Use a signal emitted right after the load-changed with committed > providing both the certificate and tls-errors as parameters. > WebKitWebView::received-certificate or something like that. > > b) Before emitting the load-changed signal with committed, we set the > certificate and tls-errors to the WebKitURIResponse of the main > resource. And we add API to WebKitURIResponse to get the certificate and > tls-errors. The ephy code would be mostly the same with this solution. > > c) An alternative to b) would be to always serialize the certificate > info with the ResourceResponse, and the WebKitURIResponse will always be > created with the certificate and tls-errors. This would be simpler, but > we would be including the certificate in all resource responses which > could affect the IPC traffic. > > d) Save the certificate and tls-errors in the web view, similar to what > WK2 does, saving the certificate info in the frame proxy. Then we could > add api to get both from the view, or add a wrapper as I said before > containing both. > > what do you think? any other possibility? > > Allow apps to decide whether accept or deny a certificate > - > > This is not used by current ephy, but it would allow apps to implement > the custom error page that shows information about the certificate and > tls errors, and the user can ignore the errors and continue with the > load or cancel the load. I've started to work on this following Dan > suggestions, and adding support for this in WebCore. The patch has been > rejected because it's soup specific, but I will rework it to make it > available to all other ports More details here: > > https://bugs.webkit.org/show_bug.cgi?id=90267 > > Once we have the required support in WebCore, we'll send a message to > the UI process in case of ssl errors to let the webkit layer decide. In > this case I think we can use the generic permission request api that > Mario added for geolocation. We would only need to add a > WebKitTlsPermissionRequest object that implements the > WebKitPermissionRequest interface and provides API to get the > certificate and tls-errors. Then we simply emit the > WebKitWebView::permission-request signal when we receive the message > from the web process. > > Comments? > -- > Carlos Garcia Campos > http://pgp.rediris.es:11371/pks/lookup?op=get&search=0xF3D322D0EC4582C3 > ___ > webkit-gtk mailing list > webkit-gtk@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] Cookies API in WebKit2
On Mon, Mar 26, 2012 at 11:20:05AM +0200, Sergio Villar Senin wrote:
> En 26/03/12 10:40, Carlos Garcia Campos escribiu:
> > typedef enum { WEBKIT_COOKIE_STORAGE_TEXT,
> > WEBKIT_COOKIE_STORAGE_SQLITE } WebKitCookieStorage;
> >
> > void webkit_cookie_manager_set_storage (WebKitCookieManager
> > *cookie_manager, const gchar *storage_path,
> > WebKitCookieStoragestorage);
>
> Do we really need this API?. I mean why a client would have to care
> about this? I guess we should use the sqlite backend by default and
> fallback to text storage if needed.
I use this in my browser and even with webkit 1 API there is not enough
granularity by farting directly with the libsoup cookie jar. Taking
away more features will make this only harder. Currently I jump through
some hoops to implement cookie whitelisting inside xxxterm. I am
thinking that webkit2 is going to take away this option completely
without making direct changes to webkit and maintaining my own.
I honestly have been following the webkit2 API with some horror because
it is giving me less control over fundamental pieces to implement
privacy control features. I fear that webkit2 is heading in the wrong
direction regarding privacy protection.
FWIW,
/marco
>
> BR
> ___
> webkit-gtk mailing list
> webkit-gtk@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] Changeset 95567 fixes hang
Awesome! On Mon, Feb 06, 2012 at 05:15:06PM -0800, Martin Robinson wrote: > On Mon, Feb 6, 2012 at 5:09 PM, Marco Peereboom wrote: > > Can we please get http://trac.webkit.org/changeset/95567/ merged in the > > 1.6 tree? ??This issue hits everyone who uses twitter; if one clicks on > > the little icon on the top right corner (the < or > button) webkit loops > > forever because it doesn't have a proper termination condition. ??I can > > confirm that this bug fixes it on 1.6 trees. > > I've added it here: https://trac.webkit.org/wiki/WebKitGTK/1.6.x > > When the next version is released, that fix should be there. > > --Martin ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
[webkit-gtk] Changeset 95567 fixes hang
Can we please get http://trac.webkit.org/changeset/95567/ merged in the 1.6 tree? This issue hits everyone who uses twitter; if one clicks on the little icon on the top right corner (the < or > button) webkit loops forever because it doesn't have a proper termination condition. I can confirm that this bug fixes it on 1.6 trees. Thanks, /marco ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] webkit gtk webinspector
It works over here. I suspect something went wrong during
compilation/install.
On Wed, Dec 07, 2011 at 01:04:38AM +0900, zhen ye gan wrote:
>Hi,
>I've compiled a webkit gtk with the stable release code from webkitgtk+
>(webkit-1.6.1) on Ubuntu 11.10.
>I launched the GtkLauncher with --enable-developer-extras=true, but when i
>clicked on the inspect element button, nothing came out.
>I think it's because of InspectorClientGtk.cpp where g_signal_emit_by_name
>returned NULL to the inspectorWebView.
>WebKitWebView* inspectorWebView = 0;
>g_signal_emit_by_name(webInspector, "inspect-web-view",
>m_inspectedWebView, &inspectorWebView);
>if (!inspectorWebView) {
>g_object_unref(webInspector);
>return;
>}
>My question is why does it return a null value and how do i solve it?
>Thanks,
>zy
> ___
> webkit-gtk mailing list
> webkit-gtk@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] webkit gtk webinspector
Oops the last email I sent was not sent to the correct person. Please
disregard and sorry for the noise.
On Wed, Dec 07, 2011 at 01:04:38AM +0900, zhen ye gan wrote:
>Hi,
>I've compiled a webkit gtk with the stable release code from webkitgtk+
>(webkit-1.6.1) on Ubuntu 11.10.
>I launched the GtkLauncher with --enable-developer-extras=true, but when i
>clicked on the inspect element button, nothing came out.
>I think it's because of InspectorClientGtk.cpp where g_signal_emit_by_name
>returned NULL to the inspectorWebView.
>WebKitWebView* inspectorWebView = 0;
>g_signal_emit_by_name(webInspector, "inspect-web-view",
>m_inspectedWebView, &inspectorWebView);
>if (!inspectorWebView) {
>g_object_unref(webInspector);
>return;
>}
>My question is why does it return a null value and how do i solve it?
>Thanks,
>zy
> ___
> webkit-gtk mailing list
> webkit-gtk@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] webkit_dom_* api
On Tue, Nov 08, 2011 at 11:30:18AM -0600, Marco Peereboom wrote: > I am working on some code on my xxxterm browser and have come to the > point that I need to be able to touch the DOM stuff. I don't seem to be > able to find any API docs and have been trolling the code instead. > > I have 2 questions. > 1. Are there any docs or is the source + UT the doco? > 2. Can someone tell me the magic to know what DOM node is active on a >webpage? I want to be able to determine if the browser is sitting in >a user editable field in order to handle keystrokes differently. I actually found it and I'll paste it here in the hopes to save a poor soul the same amount of time I spent on it. WebKitDOMDocument *doc; WebKitDOMElement*active; doc = webkit_web_view_get_dom_document(webview); active = webkit_dom_html_document_get_active_element((WebKitDOMHTMLDocument*)doc); > On #2 I looked at some JS code however I prefer to use C code for this > for a number of reasons. > > Thanks, > /marco > ___ > webkit-gtk mailing list > webkit-gtk@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
[webkit-gtk] webkit_dom_* api
I am working on some code on my xxxterm browser and have come to the point that I need to be able to touch the DOM stuff. I don't seem to be able to find any API docs and have been trolling the code instead. I have 2 questions. 1. Are there any docs or is the source + UT the doco? 2. Can someone tell me the magic to know what DOM node is active on a webpage? I want to be able to determine if the browser is sitting in a user editable field in order to handle keystrokes differently. On #2 I looked at some JS code however I prefer to use C code for this for a number of reasons. Thanks, /marco ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
[webkit-gtk] Setting to block displaying and/or running insecure content on secure pages
While researching the BEAST exploit I did some tests and found out that webkit is susceptible to one of the preconditions of the exploit. Namely the downloading and executing scripts and content outside of the https session. Now we can argue the likelihood of the BEAST exploit in webkit but that isn't my point at all (I personally think it is exceedingly unlikely). The browser that I wrote uses whitelists to determine what domains are allowed to run javascript. By default webkit allows running of cross domain scripts so when I whitelist a domain a site currently can and will download scripts from another domain I might not trust. I did a proof of concept with the following web page: https://www.cyphertite.com/beasttest.html It downloads a picture and resizes it when JS is enabled even though I don't trust the domain where the picture and JS originated. Speaking as a proof of concept of course ;-) I rigged webkit to set m_allowRunningOfInsecureContent and m_allowDisplayOfInsecureContent to false and the results were exactly what I was expecting and looking for. So I went ahead and started writing a patch to expose those settings and as I was doing that I ran across this bug report and patch: https://bugs.webkit.org/show_bug.cgi?id=58378 It exposes the settings on all platforms, perfect ;-) I would even argue that these settings should be disabled by default because I can only come up with 3 reasons to enable it and all of them are nefarious. So allowing browser authors to make this decision for themselves is therefore the best next thing IMO. Thanks for considering this. ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
On Fri, Jul 08, 2011 at 02:46:20PM -0500, Marco Peereboom wrote:
> This seems to be fixable by doing something like this:
> --- Source/WebCore/platform/graphics/FontFallbackList.cpp.origFri Jul
> 8 14:05:45 2011
> +++ Source/WebCore/platform/graphics/FontFallbackList.cpp Fri Jul 8
> 14:06:48 2011
> @@ -73,7 +73,14 @@ void FontFallbackList::releaseFontData()
>
> void FontFallbackList::determinePitch(const Font* font) const
> {
> +if (font == NULL)
> + return;
> +
> const FontData* fontData = primaryFontData(font);
> +
> +if (fontData == NULL)
> + return;
> +
> if (!fontData->isSegmented())
> m_pitch = static_cast(fontData)->pitch();
> else {
>
https://bugs.webkit.org/show_bug.cgi?id=64364
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
https://bugs.webkit.org/show_bug.cgi?id=64355 On Mon, Jul 11, 2011 at 12:16:28PM +1200, Gavin Lambert wrote: > Quoth Marco Peereboom: > > So I found the bug that causes this however I do not have a patch. > > What happens is that in my browser I set max-conns and max-conns-per-host > > right after obtaining the default session. Like so: > > > > session = webkit_get_default_session(); > > g_object_set(session, "max-conns", max_connections, (char > > *)NULL); > > g_object_set(session, "max-conns-per-host", max_host_connections, > > (char *)NULL); > > > > Problem is that webkit will overwrite these values upon "first contact" > > with a site. The responsible file is: > > WebCore/platform/network/soup/ResourceHandleSoup.cpp in function > > static void ensureSessionIsInitialized(SoupSession* session) > > static const int maxConnections = 60; > > static const int maxConnectionsPerHost = 6; > > ... > > g_object_set(session, > > SOUP_SESSION_MAX_CONNS, maxConnections, > > SOUP_SESSION_MAX_CONNS_PER_HOST, > > maxConnectionsPerHost, > > NULL); > > > > This code is run AFTER I set those values. I am drawing a blank on how > > to initialize webkit to a point where I can overwrite those values with > > something that makes sense in my environment. > > I would think that the obvious solution would be to move these calls to the > internal implementation of webkit_get_default_session, so that they'd apply > in the normal case but a custom handler can override them. (I haven't > looked at the code internals, though, so this might be more complicated than > it sounds.) > > ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
On Thu, Jun 30, 2011 at 09:54:34AM -0500, Marco Peereboom wrote:
> On Wed, Jun 29, 2011 at 06:27:30PM -0500, Marco Peereboom wrote:
> > On Wed, Jun 29, 2011 at 04:23:00PM -0700, Martin Robinson wrote:
> > > On Wed, Jun 29, 2011 at 2:43 PM, Marco Peereboom
> > > wrote:
> > >
> > > > This is with the webkit 1.4.1 snapshot with web sockets enabled. ??By
> > > > default xxxterm sets max-conns to 25 and max-conns-per-host to 5. ??When
> > > > webkit doesn't crash I see as many as 119 connections to port 80 in
> > > > various states (TIME_WAIT, ESTABLISHED, CLOSED_WAIT, SYN_SENT etc).
> > > > After a while the connections drop to 47 and are all in
> > > > ESTABLISHED_STATE. ??This number is obviously much higher than the limit
> > > > that is set to 25. ??With webit 1.2.7 I don't see this behavior.
> > >
> > > Do you mind confirming whether this behavior persists with WebSockets
> > > disabled?
> >
> > Nope not at all. I have toyed but do not have a detailed report. My
> > non-scientific test was "better" but not gone. For example the
> > CLOSED_WAIT number seems to be gone completely but it still breaches the
> > max-conns setting and I see similar crashes. I will do the build and
> > generate an actual report.
>
> Ok, the results are as I described with web sockets disabled. Better as
> in webkit does close file descriptors much quicker however the
> fundamental issues of breaching the max-conns limit is still there.
So I found the bug that causes this however I do not have a patch. What
happens is that in my browser I set max-conns and max-conns-per-host
right after obtaining the default session. Like so:
session = webkit_get_default_session();
g_object_set(session, "max-conns", max_connections, (char *)NULL);
g_object_set(session, "max-conns-per-host", max_host_connections,
(char *)NULL);
Problem is that webkit will overwrite these values upon "first contact"
with a site. The responsible file is:
WebCore/platform/network/soup/ResourceHandleSoup.cpp in function
static void ensureSessionIsInitialized(SoupSession* session)
static const int maxConnections = 60;
static const int maxConnectionsPerHost = 6;
...
g_object_set(session,
SOUP_SESSION_MAX_CONNS, maxConnections,
SOUP_SESSION_MAX_CONNS_PER_HOST, maxConnectionsPerHost,
NULL);
This code is run AFTER I set those values. I am drawing a blank on how
to initialize webkit to a point where I can overwrite those values with
something that makes sense in my environment.
Any clues?
> When I launch the browser with all those links at once it opens up about
> 67 file descriptors and none end up in CLOSED_WAIT initially. Instead
> one sees ESTABLISHED and TIME_WAIT. In other words when running without
> web sockets the inevitable is just postponed a little. A couple of
> quick restarts and webkit exhibits the same crashes. I do see the same
> misrendered pages as described before, even when starting on a fresh
> machine (virtually no FDs in use).
This seems to be fixable by doing something like this:
--- Source/WebCore/platform/graphics/FontFallbackList.cpp.orig Fri Jul 8
14:05:45 2011
+++ Source/WebCore/platform/graphics/FontFallbackList.cpp Fri Jul 8
14:06:48 2011
@@ -73,7 +73,14 @@ void FontFallbackList::releaseFontData()
void FontFallbackList::determinePitch(const Font* font) const
{
+if (font == NULL)
+ return;
+
const FontData* fontData = primaryFontData(font);
+
+if (fontData == NULL)
+ return;
+
if (!fontData->isSegmented())
m_pitch = static_cast(fontData)->pitch();
else {
> For completions sake, this is using webkit 1.4.1 and libsoup 2.34.2 and
> I do not see these issues when I used webkit 1.2.x and libsoup 2.32.y.
> I also validated that I see these same issues using other browsers such
> as epiphany and midori.
>
> I'll be happy to run some additional tests or dig in the code if you
> point me in a potential direction.
>
> >
> > >
> > > --Martin
I see some more issues and will write separate emails for those.
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
On Thu, Jun 30, 2011 at 09:54:34AM -0500, Marco Peereboom wrote:
> On Wed, Jun 29, 2011 at 06:27:30PM -0500, Marco Peereboom wrote:
> > On Wed, Jun 29, 2011 at 04:23:00PM -0700, Martin Robinson wrote:
> > > On Wed, Jun 29, 2011 at 2:43 PM, Marco Peereboom
> > > wrote:
> > >
> > > > This is with the webkit 1.4.1 snapshot with web sockets enabled. ??By
> > > > default xxxterm sets max-conns to 25 and max-conns-per-host to 5. ??When
> > > > webkit doesn't crash I see as many as 119 connections to port 80 in
> > > > various states (TIME_WAIT, ESTABLISHED, CLOSED_WAIT, SYN_SENT etc).
> > > > After a while the connections drop to 47 and are all in
> > > > ESTABLISHED_STATE. ??This number is obviously much higher than the limit
> > > > that is set to 25. ??With webit 1.2.7 I don't see this behavior.
> > >
> > > Do you mind confirming whether this behavior persists with WebSockets
> > > disabled?
> >
> > Nope not at all. I have toyed but do not have a detailed report. My
> > non-scientific test was "better" but not gone. For example the
> > CLOSED_WAIT number seems to be gone completely but it still breaches the
> > max-conns setting and I see similar crashes. I will do the build and
> > generate an actual report.
>
> Ok, the results are as I described with web sockets disabled. Better as
> in webkit does close file descriptors much quicker however the
> fundamental issues of breaching the max-conns limit is still there.
>
> When I launch the browser with all those links at once it opens up about
> 67 file descriptors and none end up in CLOSED_WAIT initially. Instead
> one sees ESTABLISHED and TIME_WAIT. In other words when running without
> web sockets the inevitable is just postponed a little. A couple of
> quick restarts and webkit exhibits the same crashes. I do see the same
> misrendered pages as described before, even when starting on a fresh
> machine (virtually no FDs in use).
>
> For completions sake, this is using webkit 1.4.1 and libsoup 2.34.2 and
> I do not see these issues when I used webkit 1.2.x and libsoup 2.32.y.
> I also validated that I see these same issues using other browsers such
> as epiphany and midori.
>
> I'll be happy to run some additional tests or dig in the code if you
> point me in a potential direction.
One more trace too:
#0 0x0002120e1b17 in WTF::cryptographicallyRandomValuesFromOS () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#1 0x0002120cee7b in WTF::(anonymous
namespace)::ARC4RandomNumberGenerator::stirIfNeeded () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#2 0x0002120cf011 in WTF::cryptographicallyRandomNumber () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#3 0x0002120e1c79 in WTF::randomNumber () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#4 0x000211a3 in WebCore::JSDOMGlobalObject::JSDOMGlobalObject () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#5 0x00021144637b in WebCore::JSDOMWindowBase::JSDOMWindowBase () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#6 0x000211c81763 in WebCore::JSDOMWindow::JSDOMWindow () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#7 0x00021144e76a in WebCore::JSDOMWindowShell::setWindow () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#8 0x00021144eafa in WebCore::JSDOMWindowShell::JSDOMWindowShell () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#9 0x00021148589b in WebCore::ScriptController::createWindowShell () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#10 0x000211485ce3 in WebCore::ScriptController::initScript () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#11 0x00021136819b in webkit_web_frame_get_global_context () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#12 0x0040f2d8 in run_script (t=0x2088d0400, s=0x519be8 "/*(c)2009 by
Leon Winter(c)2009,2010 by Hannes Schueller(c)2010 by Hans-Peter Deifel see
LICENSE file*/ function
vimprobable_clearfocus(){if(document.activeElement&&document.activeElement.blur)documen"...)
at /home/marco/xxxterm/xxxterm.c:1918
#13 0x00020983a06f in g_closure_invoke () from
/usr/local/lib/libgobject-2.0.so.2800.0
#14 0x0002098511e1 in g_signal_handlers_block_matched () from
/usr/local/lib/libgobject-2.0.so.2800.0
#15 0x0002098530f5 in g_signal_emit_valist () from
/usr/local/lib/libgobject-2.0.so.2800.0
#16 0x0002098533b8 in g_signal_emit_by_name () from
/usr/local/lib/libgobject-2.0.so.2800.0
#17 0x000211848642 in WebCore::ProgressTracker::finalProgressComplete ()
from /usr/local/lib/libwebkitgtk-1.0.so.0.1
#18 0x000211848708 in WebCore::ProgressTracker::progressCompleted () from
/usr/local/lib/libwebkitgtk-1.0.so.0.1
#19 0x000211818ce5 in WebCore::FrameLoader::recursiveCheckLoadComplet
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
On Wed, Jun 29, 2011 at 06:27:30PM -0500, Marco Peereboom wrote: > On Wed, Jun 29, 2011 at 04:23:00PM -0700, Martin Robinson wrote: > > On Wed, Jun 29, 2011 at 2:43 PM, Marco Peereboom wrote: > > > > > This is with the webkit 1.4.1 snapshot with web sockets enabled. ??By > > > default xxxterm sets max-conns to 25 and max-conns-per-host to 5. ??When > > > webkit doesn't crash I see as many as 119 connections to port 80 in > > > various states (TIME_WAIT, ESTABLISHED, CLOSED_WAIT, SYN_SENT etc). > > > After a while the connections drop to 47 and are all in > > > ESTABLISHED_STATE. ??This number is obviously much higher than the limit > > > that is set to 25. ??With webit 1.2.7 I don't see this behavior. > > > > Do you mind confirming whether this behavior persists with WebSockets > > disabled? > > Nope not at all. I have toyed but do not have a detailed report. My > non-scientific test was "better" but not gone. For example the > CLOSED_WAIT number seems to be gone completely but it still breaches the > max-conns setting and I see similar crashes. I will do the build and > generate an actual report. Ok, the results are as I described with web sockets disabled. Better as in webkit does close file descriptors much quicker however the fundamental issues of breaching the max-conns limit is still there. When I launch the browser with all those links at once it opens up about 67 file descriptors and none end up in CLOSED_WAIT initially. Instead one sees ESTABLISHED and TIME_WAIT. In other words when running without web sockets the inevitable is just postponed a little. A couple of quick restarts and webkit exhibits the same crashes. I do see the same misrendered pages as described before, even when starting on a fresh machine (virtually no FDs in use). For completions sake, this is using webkit 1.4.1 and libsoup 2.34.2 and I do not see these issues when I used webkit 1.2.x and libsoup 2.32.y. I also validated that I see these same issues using other browsers such as epiphany and midori. I'll be happy to run some additional tests or dig in the code if you point me in a potential direction. > > > > > --Martin > ___ > webkit-gtk mailing list > webkit-gtk@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
On Wed, Jun 29, 2011 at 04:23:00PM -0700, Martin Robinson wrote: > On Wed, Jun 29, 2011 at 2:43 PM, Marco Peereboom wrote: > > > This is with the webkit 1.4.1 snapshot with web sockets enabled. ??By > > default xxxterm sets max-conns to 25 and max-conns-per-host to 5. ??When > > webkit doesn't crash I see as many as 119 connections to port 80 in > > various states (TIME_WAIT, ESTABLISHED, CLOSED_WAIT, SYN_SENT etc). > > After a while the connections drop to 47 and are all in > > ESTABLISHED_STATE. ??This number is obviously much higher than the limit > > that is set to 25. ??With webit 1.2.7 I don't see this behavior. > > Do you mind confirming whether this behavior persists with WebSockets > disabled? Nope not at all. I have toyed but do not have a detailed report. My non-scientific test was "better" but not gone. For example the CLOSED_WAIT number seems to be gone completely but it still breaches the max-conns setting and I see similar crashes. I will do the build and generate an actual report. > > --Martin ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
Ok I captured a few screenshots to illustrate what I see and some backtraces as well when I push file descriptors well past starvation. The test I run is always the same. I open a session which in the xxxterm browser means "an X number of tabs with links in them". The links are: https://www.cyphertite.com/ http://arstechnica.com/ http://www.wired.com/ http://www.huffingtonpost.com/ http://slashdot.org/ http://www.physorg.com/ http://sportsillustrated.cnn.com/ http://espn.go.com/ about half were rendered correctly and half didn't. Missing CSS and pictures: http://opensource.conformal.com/snapshots/tmp/arstechnica.png http://opensource.conformal.com/snapshots/tmp/cyphertite.png http://opensource.conformal.com/snapshots/tmp/slashdot.png Can't resolve hostname: http://opensource.conformal.com/snapshots/tmp/cnnsi.png http://opensource.conformal.com/snapshots/tmp/espn.png Repeating the test of opening 8 tabs at once often leads to crashes as well. For example: #0 0x0002110c8d7b in WebCore::FontFallbackList::determinePitch () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #1 0x0002111804be in WebCore::RenderBlock::findNextLineBreak () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #2 0x000211181c4c in WebCore::RenderBlock::layoutInlineChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #3 0x000211170bf8 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #4 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #5 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #6 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #7 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #8 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #9 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #10 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #11 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #12 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #13 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #14 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #15 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #16 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #17 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #18 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #19 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #20 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #21 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #22 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #23 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #24 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #25 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #26 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #27 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #28 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #29 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #30 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #31 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #32 0x000211159fbd in WebCore::RenderBlock::layout () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #33 0x00021116e552 in WebCore::RenderBlock::layoutBlockChild () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #34 0x0002111701b8 in WebCore::RenderBlock::layoutBlockChildren () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #35 0x0002111711d6 in WebCore::RenderBlock::layoutBlock () from /usr/local/lib/libwebkitgtk-1.0.so.0.1 #36 0x000211159fbd in WebCore::RenderBlock:
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
On Mon, Jun 27, 2011 at 12:46:31PM -0400, David Hill wrote: > On Mon, Jun 27, 2011 at 12:19:08PM -0400, Dan Winship wrote: > :On 06/24/2011 04:49 PM, David Hill wrote: > :> When using a browser with WebkitGTk 1.4.1, I get a lot of CLOSE_WAITs > :> sitting around. Does Webkit open up sockets itself, or does it solely > :> use libsoup to do the web connections? > : > :This is one of the leftover bits of historical lameness in libsoup. It > :doesn't pay attention to persistent connections when they're idle. So if > :the server times out a connection and closes it, it will show up in > :CLOSE_WAIT on the client for a while. Libsoup will only notice it when > :either you try to connect to that server again, or if it hits the > :maximum-number-of-open-connections limit. > : > :-- Dan > > It seems if I compile webkit 1.4.1 with --enable-web-sockets=no, the > CLOSE_WAIT issues go away. I tried that too however max connections still seems to not be honored. ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] 1.4.1 - CLOSE_WAIT
On Mon, Jun 27, 2011 at 12:19:08PM -0400, Dan Winship wrote: > On 06/24/2011 04:49 PM, David Hill wrote: > > When using a browser with WebkitGTk 1.4.1, I get a lot of CLOSE_WAITs > > sitting around. Does Webkit open up sockets itself, or does it solely > > use libsoup to do the web connections? > > This is one of the leftover bits of historical lameness in libsoup. It > doesn't pay attention to persistent connections when they're idle. So if > the server times out a connection and closes it, it will show up in > CLOSE_WAIT on the client for a while. Libsoup will only notice it when > either you try to connect to that server again, or if it hits the > maximum-number-of-open-connections limit. Actually I don't think this number is honored at all. When I set max connections to 10 I see like 68 open file descriptors in one of my tests (open a browser with a bunch of tabs). I suspect that issues I see using webkit 1.4 are mostly related to file descriptor starvation. I did not have this issue at all with the 1.2 tree. The symptoms I see are crashes, misrendered web pages, missing CSS, missing favicon, missing pictures etc. I am running some more tests before I can make a more educated assessment but if someone has a clue bat I'll take it. Would merging these patches help? https://bugs.webkit.org/show_bug.cgi?id=35721 https://bugs.webkit.org/show_bug.cgi?id=35573 > > -- Dan > ___ > webkit-gtk mailing list > webkit-gtk@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] Preparing 1.4.0. What to merge
I really would like to see 80856 go in. http://trac.webkit.org/changeset/80856 Thanks, /marco ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
[webkit-gtk] webkit 1.2.7 crashes when visiting a site
When visiting http://design.canonical.com/2011/03/quit/ webkit 1.2.7 crashes with the following backtrace: #0 0x000210eb25c0 in WebCore::ImageBuffer::context () from /usr/local/lib/libwebkit-1.0.so.2.2 #1 0x000210eb186d in WebCore::drawPathShadow () from /usr/local/lib/libwebkit-1.0.so.2.2 #2 0x000210eb1e5a in WebCore::GraphicsContext::fillRoundedRect () from /usr/local/lib/libwebkit-1.0.so.2.2 #3 0x000210cb95bc in WebCore::RenderBoxModelObject::paintBoxShadow () from /usr/local/lib/libwebkit-1.0.so.2.2 #4 0x000210cb1453 in WebCore::RenderBox::paintBoxDecorations () from /usr/local/lib/libwebkit-1.0.so.2.2 #5 0x000210c9a7e6 in WebCore::RenderBlock::paintObject () from /usr/local/lib/libwebkit-1.0.so.2.2 #6 0x000210c8ea79 in WebCore::RenderBlock::paint () from /usr/local/lib/libwebkit-1.0.so.2.2 #7 0x000210ce8fad in WebCore::RenderLayer::paintLayer () from /usr/local/lib/libwebkit-1.0.so.2.2 #8 0x000210ce8cd1 in WebCore::RenderLayer::paintLayer () from /usr/local/lib/libwebkit-1.0.so.2.2 #9 0x000210ce8cd1 in WebCore::RenderLayer::paintLayer () from /usr/local/lib/libwebkit-1.0.so.2.2 #10 0x000210ce94a4 in WebCore::RenderLayer::paint () from /usr/local/lib/libwebkit-1.0.so.2.2 #11 0x000210bd6ea1 in WebCore::FrameView::paintContents () from /usr/local/lib/libwebkit-1.0.so.2.2 #12 0x000210c1c49e in WebCore::ScrollView::paint () from /usr/local/lib/libwebkit-1.0.so.2.2 #13 0x000210f10b0d in webkit_web_view_expose_event () from /usr/local/lib/libwebkit-1.0.so.2.2 #14 0x00020f131d2f in gtk_marshal_BOOLEAN__VOID () from /usr/local/lib/libgtk-x11-2.0.so.2200.0 #15 0x000208864f6f in g_closure_invoke () from /usr/local/lib/libgobject-2.0.so.2600.0 #16 0x00020887c1db in g_signal_handlers_block_matched () from /usr/local/lib/libgobject-2.0.so.2600.0 #17 0x00020887dbdf in g_signal_emit_valist () from /usr/local/lib/libgobject-2.0.so.2600.0 #18 0x00020887e323 in g_signal_emit () from /usr/local/lib/libgobject-2.0.so.2600.0 #19 0x00020f243acf in gtk_widget_style_attach () from /usr/local/lib/libgtk-x11-2.0.so.2200.0 #20 0x00020f12bca3 in gtk_main_do_event () from /usr/local/lib/libgtk-x11-2.0.so.2200.0 #21 0x000211f1bc53 in gdk_window_invalidate_region () from /usr/local/lib/libgdk-x11-2.0.so.2200.0 #22 0x000211f1bc93 in gdk_window_invalidate_region () from /usr/local/lib/libgdk-x11-2.0.so.2200.0 #23 0x000211f17cb4 in gdk_window_is_viewable () from /usr/local/lib/libgdk-x11-2.0.so.2200.0 #24 0x000211f19dee in gdk_window_process_all_updates () from /usr/local/lib/libgdk-x11-2.0.so.2200.0 #25 0x00020f0aeed1 in gtk_container_check_resize () from /usr/local/lib/libgtk-x11-2.0.so.2200.0 #26 0x000211ef6aae in gdk_threads_add_timeout_seconds () from /usr/local/lib/libgdk-x11-2.0.so.2200.0 #27 0x00020b458b93 in g_main_context_dispatch () from /usr/local/lib/libglib-2.0.so.2600.0 #28 0x00020b45c460 in g_main_context_prepare () from /usr/local/lib/libglib-2.0.so.2600.0 #29 0x00020b45c855 in g_main_loop_run () from /usr/local/lib/libglib-2.0.so.2600.0 #30 0x00020f12be93 in gtk_main () from /usr/local/lib/libgtk-x11-2.0.so.2200.0 #31 0x00416b7d in main (argc=0, argv=0x7f7ea4a0) at /home/marco/src/xxxterm/xxxterm.c:8042 Worth writing up on the webkit bugzilla? Anyone else seeing this? ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] dns prefetching knob
On Wed, Mar 09, 2011 at 10:30:00AM -0800, Evan Martin wrote: > On Wed, Mar 9, 2011 at 6:18 AM, David Hill wrote: > >> Prefetch isn't mentioned in the HTTP spec at all. And I don't think > >> anyone was doing it back in the modem age. AFAIK, Google started doing > >> it with Chrome, and other people followed. > > > > If it is not mentioned in the spec, I think it should be optionized > > then, not hardcoded. ?Google started doing it with Chrome, but was it a > > performance booster, or (imo) a way to track users even more? > > Your conspiracy theory is impressive, but as far as I know Google > doesn't get to observe the DNS traffic of every person on the > internet. Maybe you know something about DNS that I don't. ;) http://code.google.com/speed/public-dns/ just sayin' :-) > We did measure the benefit of DNS prefetching. When we prefetch a > hostname, we record the time that we needed to wait for the resolution > to complete. When we later do a real fetch for that host, we record > that the time we spent prefetching was a gain. You can see these > numbers in Chrome by visiting > about:histograms/DNS.ResolveSuccess > > For users who have opted in to reporting anonymous statistics, we send > those numbers (no hostnames, just the millisecond counts) to Google. > Looking at that data right now, I see that 10% of users' requests were > sped up by 400ms or more, which is huge in terms of web speed. > (Consider that you can't even start fetching subresources until you've > successfully made the initial connection the host.) > There's a screenshot of some older data here: > http://blog.chromium.org/2008/09/dns-prefetching-or-pre-resolving.html Not sure I read the results correctly but I seem to only see raw DNS lookup speed gain. Not overall "user visible page load times". So since I am not sure I get the results I won't comment further on it. > > > I have tested this patch and the performance boost is unbelievable. > > I guess on IRC Marco was saying that DNS on OpenBSD is single Uhm, I don't recall being on IRC ;-) > threaded? Maybe I misunderstood. It would sound like a bug in > libsoup to me if so. Certainly if you can't resolve DNS in parallel I > would expect it's better to do fewer resolutions. The OpenBSD libc resolver is a single pipe and it obviously exacerbates the problem however, I use a resolver that parallelizes things. Unfortunately I seem to mostly use really crappy nameservers so I see issues all the time. > For the record, I fully support you adding the option to disable > prefetching. I just hope that you want to do it for a good reason. Sure, we agree here. I have a number of reasons from performance to security to paranoia to bandwitdh etc. The fact that I am paranoid doesn't mean they aren't watching :-) ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] dns prefetching knob
On Wed, Mar 09, 2011 at 08:55:37AM -0800, Martin Robinson wrote:
> On Wed, Mar 9, 2011 at 6:28 AM, Marco Peereboom wrote:
> > Hence the knobbing part. ??Some people have legitimate use for it which
> > is of course fine. ??I want it for performance reasons but also to
> > prevent DNS tracking. ??Between DNS prefetching, link hovering and the
> > empty DNS lookups webkit generates one can get near pixel resolution on
> > how the user moves his mouse. ??This comes in handy for "Online
> > Behavioral Tracking". ??Being a practical paranoid I'd like to prevent
> > that.
>
> Is it possible for you to make the tools you used to generate your
> performance number available publically? This would help us fix any
> performance issues that exist as bugs. This may very well be some
> weird corner case that we're hitting in libsoup. Let's fix it!
Sure, it is far less interesting than you'd hope for ;-)
The patch at the bottom is what generates the numbers I quoted. It goes
against the in tree cvs version of xxxterm.
Tree is at:
CVSROOT=anon...@opensource.conformal.com:/anoncvs/xxxterm
The DNS lookup stuff I measured with the adsuck stats but I don't think
that is pertinent to this discussion. If you guys want a rundown on
that as well let me know and I'll show ya.
> I do not think there is any harm in exposing this setting. I do
> believe that those pushing for it should try to clarify what it is
> though. From what I gather it's effectively two pieces:
>
> 1. A backport of a Settings.cpp DNS prefetch change to the stable branch.
Correct.
> 2. New code which exposes this setting in the WebKitWebSettings WebKitGTK+
> API.
Right.
> Really it should just be the change to WebKitWebSettings and a request
> to merge both changes back into the stable branch. Be aware that we
> are desperately close to cutting the stable branch for 1.4.x.
This is why I really really really would like to get this cranking :-)
> --Martin
Index: xxxterm.c
===
RCS file: /cvs/xxxterm/xxxterm/xxxterm.c,v
retrieving revision 1.348
diff -u -p -u -p -r1.348 xxxterm.c
--- xxxterm.c 8 Mar 2011 23:19:36 - 1.348
+++ xxxterm.c 9 Mar 2011 17:38:34 -
@@ -218,6 +218,9 @@ struct tab {
WebKitWebSettings *settings;
int font_size;
gchar *user_agent;
+
+ struct timeval sod;
+ struct timeval eod;
};
TAILQ_HEAD(tab_list, tab);
@@ -492,6 +495,7 @@ char*user_agent = NULL;
intsave_rejected_cookies = 0;
time_t session_autosave = 0;
intguess_search = 0;
+intdns_prefetch = FALSE;
struct settings;
struct key_binding;
@@ -5768,6 +5772,7 @@ notify_load_status_cb(WebKitWebView* wvi
struct history *h, find;
const gchar *s_loading;
struct karg a;
+ struct timeval elapsed;
DNPRINTF(XT_D_URL, "notify_load_status_cb: %d\n",
webkit_web_view_get_load_status(wview));
@@ -5780,6 +5785,7 @@ notify_load_status_cb(WebKitWebView* wvi
switch (webkit_web_view_get_load_status(wview)) {
case WEBKIT_LOAD_PROVISIONAL:
/* 0 */
+ gettimeofday(&t->sod, NULL);
abort_favicon_download(t);
#if GTK_CHECK_VERSION(2, 20, 0)
gtk_widget_show(t->spinner);
@@ -5865,6 +5871,11 @@ notify_load_status_cb(WebKitWebView* wvi
gtk_label_set_text(GTK_LABEL(t->label), "(untitled)");
default:
gtk_widget_set_sensitive(GTK_WIDGET(t->stop), FALSE);
+
+ gettimeofday(&t->eod, NULL);
+ timersub(&t->eod, &t->sod, &elapsed);
+ fprintf(stderr, "prefetch %d: page load time: %ld.%.6ld ->
%s\n",
+ dns_prefetch, elapsed.tv_sec, elapsed.tv_usec, uri);
break;
}
@@ -6618,6 +6629,8 @@ stop_cb(GtkWidget *w, struct tab *t)
void
setup_webkit(struct tab *t)
{
+ g_object_set(G_OBJECT(t->settings),
+ "dns-prefetching", dns_prefetch, (char *)NULL);
g_object_set(G_OBJECT(t->settings),
"user-agent", t->user_agent, (char *)NULL);
g_object_set(G_OBJECT(t->settings),
___
webkit-gtk mailing list
webkit-gtk@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] dns prefetching knob
On Wed, Mar 09, 2011 at 09:10:16AM -0500, Dan Winship wrote: > On 03/09/2011 07:55 AM, Marco Peereboom wrote: > > For some reason > > people are still attached to this relic of the modem era. DNS prefetch > > is optional in the HTTP spec and the time has come to admit that it no > > longer serves a purpose. Back in the day when modems were king and DNS > > lookups were expensive you could save some time when a user clicked on a > > link. Times have changes and the web has evolved. > > Prefetch isn't mentioned in the HTTP spec at all. And I don't think > anyone was doing it back in the modem age. AFAIK, Google started doing > it with Chrome, and other people followed. I am sorry I meant the optional "dns-prefetch" and "x-dns-prefetch-control" webkit looks for and does magic with. > It *does* help in some cases. Lots of sites have lame slow DNS servers > (and lots of ISPs have lame slow recursive resolvers). We just don't > want to try to pre-resolve every single link on the page. Hence the knobbing part. Some people have legitimate use for it which is of course fine. I want it for performance reasons but also to prevent DNS tracking. Between DNS prefetching, link hovering and the empty DNS lookups webkit generates one can get near pixel resolution on how the user moves his mouse. This comes in handy for "Online Behavioral Tracking". Being a practical paranoid I'd like to prevent that. And this knob exists in 1.3.12 as well. My patch just adds it to the current stable branch and exposes it in GTK. ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] dns prefetching knob
On Wed, Mar 09, 2011 at 09:33:21AM -0300, Gustavo Noronha Silva wrote: > On Wed, 2011-03-09 at 07:16 -0500, Dan Winship wrote: > > On 03/08/2011 07:07 PM, Marco Peereboom wrote: > > > I created a bug report to add a DNS prefetch knob but I was advised to > > > also send the patch to the mailing list. > > > > > > With this knob I can make the web browser I wrote (xxxterm > > > http://opensource.conformal.com/wiki/XXXTerm ) much snappier on sites > > > with lots of embedded domain names (news.google.com for example). > > > > The entire point of DNS prefetching is that it's supposed to make the > > browser faster. If it's not doing that, then the right thing would be to > > remove it entirely, not to add a configurable "be broken / don't be > > broken" option. > > I agree, although it might make sense to make it a build-time option. > Marco, in your changelog you say that you are 'Backporting' - > backporting from what? https://bugs.webkit.org/show_bug.cgi?id=28825 The mac version. I took the original patch and added the GTK bits. ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] dns prefetching knob
On Wed, Mar 09, 2011 at 07:16:14AM -0500, Dan Winship wrote: > On 03/08/2011 07:07 PM, Marco Peereboom wrote: > > I created a bug report to add a DNS prefetch knob but I was advised to > > also send the patch to the mailing list. > > > > With this knob I can make the web browser I wrote (xxxterm > > http://opensource.conformal.com/wiki/XXXTerm ) much snappier on sites > > with lots of embedded domain names (news.google.com for example). > > The entire point of DNS prefetching is that it's supposed to make the > browser faster. If it's not doing that, then the right thing would be to > remove it entirely, not to add a configurable "be broken / don't be > broken" option. I completely agree and that is what I suggested to the OpenBSD ports people. There is a whole lot of inertia on this. For some reason people are still attached to this relic of the modem era. DNS prefetch is optional in the HTTP spec and the time has come to admit that it no longer serves a purpose. Back in the day when modems were king and DNS lookups were expensive you could save some time when a user clicked on a link. Times have changes and the web has evolved. DNS lookups are nowhere as expensive as they used to be and pages these days have hundreds or even thousands of links. The chances that a user clicks a link and saves a couple of ms does no longer outweigh the cost of numerous lookups. This also reeks of committee design :-) Trade a perceived performance issue for a real one. > But I'm pretty sure the problem is just that we're prefetching too much: > https://bugs.webkit.org/show_bug.cgi?id=41630. (Although I think it's > gotten a little better since then. I don't see the spurious "no such > host"s any more...) My tests are with a caching DNS server that spoofs crappy ad servers to NXDOMAIN. If I disable that it slows down even more. Cookies and JavaScript were also disabled. So the test is best case scenario. What blew my mind is that news.google.com prefetches 350 odd domains while loading the page. Some of those are sent in parallel but at 350 one exhausts that parallel number and it becomes a waiting game. ___ webkit-gtk mailing list webkit-gtk@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-gtk
Re: [webkit-gtk] dns prefetching knob
Updated patch that includes comments from the webkit folks.
I added some performance numbers to:
https://bugs.webkit.org/show_bug.cgi?id=55973
Any chance this can make it in 1.2.8?
--- WebCore/ChangeLog.orig Tue Mar 8 16:16:50 2011
+++ WebCore/ChangeLog Tue Mar 8 16:10:10 2011
@@ -1,3 +1,19 @@
+2011-03-08 Marco Peereboom
+
+Reviewed no one yet
+
+Backport code to enable/disable dns prefetching.
+DNS prefetching is enabled by default.
+https://bugs.webkit.org/show_bug.cgi?id=28825
+
+* dom/Document.cpp:
+(WebCore::Document::initDNSPrefetch): Check
settings->dnsPrefetchingEnabled().
+* page/Settings.cpp:
+(WebCore::Settings::Settings): Set m_dnsPrefetchingEnabled to true.
+(WebCore::Settings::setDNSPrefetchingEnabled): Added. Set
m_dnsPrefetchingEnabled.
+* page/Settings.h:
+(WebCore::Settings::dnsPrefetchingEnabled): Added. Return
m_dnsPrefetchingEnabled.
+
2010-06-21 Philippe Normand
Reviewed by Xan Lopez.
--- WebCore/dom/Document.cpp.orig Fri Sep 10 08:20:33 2010
+++ WebCore/dom/Document.cppTue Mar 8 10:16:45 2011
@@ -4639,8 +4639,10 @@ HTMLCanvasElement* Document::getCSSCanvasElement(const
void Document::initDNSPrefetch()
{
+Settings* settings = this->settings();
+
m_haveExplicitlyDisabledDNSPrefetch = false;
-m_isDNSPrefetchEnabled = securityOrigin()->protocol() == "http";
+m_isDNSPrefetchEnabled = settings && settings->dnsPrefetchingEnabled() &&
securityOrigin()->protocol() == "http";
// Inherit DNS prefetch opt-out from parent frame
if (Document* parent = parentDocument()) {
--- WebCore/page/Settings.cpp.orig Fri Sep 10 08:20:33 2010
+++ WebCore/page/Settings.cpp Tue Mar 8 15:48:38 2011
@@ -128,6 +128,7 @@ Settings::Settings(Page* page)
, m_webGLEnabled(false)
, m_loadDeferringEnabled(true)
, m_tiledBackingStoreEnabled(false)
+, m_dnsPrefetchingEnabled(true)
{
// A Frame may not have been created yet, so we initialize the
AtomicString
// hash before trying to use it.
@@ -509,6 +510,11 @@ void Settings::setShouldPaintNativeControls(bool shoul
void Settings::setUsesEncodingDetector(bool usesEncodingDetector)
{
m_usesEncodingDetector = usesEncodingDetector;
+}
+
+void Settings::setDNSPrefetchingEnabled(bool dnsPrefetchingEnabled)
+{
+m_dnsPrefetchingEnabled = dnsPrefetchingEnabled;
}
void Settings::setAllowScriptsToCloseWindows(bool allowScriptsToCloseWindows)
--- WebCore/page/Settings.h.origFri Sep 10 08:20:33 2010
+++ WebCore/page/Settings.h Tue Mar 8 10:16:45 2011
@@ -158,6 +158,9 @@ namespace WebCore {
void setUsesEncodingDetector(bool);
bool usesEncodingDetector() const { return m_usesEncodingDetector; }
+ void setDNSPrefetchingEnabled(bool);
+ bool dnsPrefetchingEnabled() const { return m_dnsPrefetchingEnabled; }
+
void setUserStyleSheetLocation(const KURL&);
const KURL& userStyleSheetLocation() const { return
m_userStyleSheetLocation; }
@@ -376,6 +379,7 @@ namespace WebCore {
bool m_webGLEnabled : 1;
bool m_loadDeferringEnabled : 1;
bool m_tiledBackingStoreEnabled : 1;
+ bool m_dnsPrefetchingEnabled : 1;
#if USE(SAFARI_THEME)
static bool gShouldPaintNativeControls;
--- WebKit/gtk/ChangeLog.orig Tue Mar 8 16:17:18 2011
+++ WebKit/gtk/ChangeLogTue Mar 8 16:14:29 2011
@@ -1,3 +1,20 @@
+2011-03-08 Marco Peereboom
+
+Reviewed no one yet
+
+Provide a knob to enable/disable DNS prefetching.
+DNS prefetching is enabled by default.
+https://bugs.webkit.org/show_bug.cgi?id=28825
+
+* WebKit/gtk/webkit/webkitwebsettings.cpp:
+(webkit_web_settings_class_init):
+(webkit_web_settings_set_property):
+(webkit_web_settings_get_property):
+(webkit_web_settings_copy):
+* webkit/webkitwebview.cpp:
+(webkit_web_view_update_settings):
+(webkit_web_view_settings_notify):
+
2010-06-15 Xan Lopez
Reviewed by Gustavo Noronha.
--- WebKit/gtk/webkit/webkitwebsettings.cpp.origTue Mar 8 11:35:14 2011
+++ WebKit/gtk/webkit/webkitwebsettings.cpp Wed Mar 9 05:43:47 2011
@@ -108,6 +108,7 @@ struct _WebKitWebSettingsPrivate {
gboolean enable_page_cache;
gboolean auto_resize_window;
gboolean enable_java_applet;
+gboolean dns_prefetching;
};
#define WEBKIT_WEB_SETTINGS_GET_PRIVATE(obj)
(G_TYPE_INSTANCE_GET_PRIVATE((obj), WEBKIT_TYPE_WEB_SETTINGS,
WebKitWebSettingsPrivate))
@@ -157,7 +158,8 @@ enum {
PROP_ENABLE_SITE_SPECIFIC_QUIRKS,
PROP_ENABLE_PAGE_CACHE,
PROP_AUTO_RESIZE_WINDOW,
-PROP_ENABLE_JAVA_APPLET
+PROP_ENABLE_JAVA_APPLET,
+PROP_DNS_PREFETCHING
};
// Create a default user agent string
@@ -857,6 +859,2
[webkit-gtk] dns prefetching knob
I created a bug report to add a DNS prefetch knob but I was advised to
also send the patch to the mailing list.
With this knob I can make the web browser I wrote (xxxterm
http://opensource.conformal.com/wiki/XXXTerm ) much snappier on sites
with lots of embedded domain names (news.google.com for example).
The diff is against the 1.2.7 snapshot
http://webkitgtk.org/webkit-1.2.7.tar.gz
Original report:
https://bugs.webkit.org/show_bug.cgi?id=55973
--- WebCore/ChangeLog.orig Tue Mar 8 16:16:50 2011
+++ WebCore/ChangeLog Tue Mar 8 16:10:10 2011
@@ -1,3 +1,19 @@
+2011-03-08 Marco Peereboom
+
+Reviewed no one yet
+
+Backport code to enable/disable dns prefetching.
+DNS prefetching is enabled by default.
+https://bugs.webkit.org/show_bug.cgi?id=28825
+
+* dom/Document.cpp:
+(WebCore::Document::initDNSPrefetch): Check
settings->dnsPrefetchingEnabled().
+* page/Settings.cpp:
+(WebCore::Settings::Settings): Set m_dnsPrefetchingEnabled to true.
+(WebCore::Settings::setDNSPrefetchingEnabled): Added. Set
m_dnsPrefetchingEnabled.
+* page/Settings.h:
+(WebCore::Settings::dnsPrefetchingEnabled): Added. Return
m_dnsPrefetchingEnabled.
+
2010-06-21 Philippe Normand
Reviewed by Xan Lopez.
--- WebCore/dom/Document.cpp.orig Fri Sep 10 08:20:33 2010
+++ WebCore/dom/Document.cppTue Mar 8 10:16:45 2011
@@ -4639,8 +4639,10 @@ HTMLCanvasElement* Document::getCSSCanvasElement(const
void Document::initDNSPrefetch()
{
+Settings* settings = this->settings();
+
m_haveExplicitlyDisabledDNSPrefetch = false;
-m_isDNSPrefetchEnabled = securityOrigin()->protocol() == "http";
+m_isDNSPrefetchEnabled = settings && settings->dnsPrefetchingEnabled() &&
securityOrigin()->protocol() == "http";
// Inherit DNS prefetch opt-out from parent frame
if (Document* parent = parentDocument()) {
--- WebCore/page/Settings.cpp.orig Fri Sep 10 08:20:33 2010
+++ WebCore/page/Settings.cpp Tue Mar 8 15:48:38 2011
@@ -128,6 +128,7 @@ Settings::Settings(Page* page)
, m_webGLEnabled(false)
, m_loadDeferringEnabled(true)
, m_tiledBackingStoreEnabled(false)
+, m_dnsPrefetchingEnabled(true)
{
// A Frame may not have been created yet, so we initialize the
AtomicString
// hash before trying to use it.
@@ -509,6 +510,11 @@ void Settings::setShouldPaintNativeControls(bool shoul
void Settings::setUsesEncodingDetector(bool usesEncodingDetector)
{
m_usesEncodingDetector = usesEncodingDetector;
+}
+
+void Settings::setDNSPrefetchingEnabled(bool dnsPrefetchingEnabled)
+{
+m_dnsPrefetchingEnabled = dnsPrefetchingEnabled;
}
void Settings::setAllowScriptsToCloseWindows(bool allowScriptsToCloseWindows)
--- WebCore/page/Settings.h.origFri Sep 10 08:20:33 2010
+++ WebCore/page/Settings.h Tue Mar 8 10:16:45 2011
@@ -158,6 +158,9 @@ namespace WebCore {
void setUsesEncodingDetector(bool);
bool usesEncodingDetector() const { return m_usesEncodingDetector; }
+ void setDNSPrefetchingEnabled(bool);
+ bool dnsPrefetchingEnabled() const { return m_dnsPrefetchingEnabled; }
+
void setUserStyleSheetLocation(const KURL&);
const KURL& userStyleSheetLocation() const { return
m_userStyleSheetLocation; }
@@ -376,6 +379,7 @@ namespace WebCore {
bool m_webGLEnabled : 1;
bool m_loadDeferringEnabled : 1;
bool m_tiledBackingStoreEnabled : 1;
+ bool m_dnsPrefetchingEnabled : 1;
#if USE(SAFARI_THEME)
static bool gShouldPaintNativeControls;
--- WebKit/gtk/ChangeLog.orig Tue Mar 8 16:17:18 2011
+++ WebKit/gtk/ChangeLogTue Mar 8 16:14:29 2011
@@ -1,3 +1,20 @@
+2011-03-08 Marco Peereboom
+
+Reviewed no one yet
+
+Provide a knob to enable/disable DNS prefetching.
+DNS prefetching is enabled by default.
+https://bugs.webkit.org/show_bug.cgi?id=28825
+
+* WebKit/gtk/webkit/webkitwebsettings.cpp:
+(webkit_web_settings_class_init):
+(webkit_web_settings_set_property):
+(webkit_web_settings_get_property):
+(webkit_web_settings_copy):
+* webkit/webkitwebview.cpp:
+(webkit_web_view_update_settings):
+(webkit_web_view_settings_notify):
+
2010-06-15 Xan Lopez
Reviewed by Gustavo Noronha.
--- WebKit/gtk/webkit/webkitwebsettings.cpp.origTue Mar 8 11:35:14 2011
+++ WebKit/gtk/webkit/webkitwebsettings.cpp Tue Mar 8 16:40:24 2011
@@ -108,6 +108,7 @@ struct _WebKitWebSettingsPrivate {
gboolean enable_page_cache;
gboolean auto_resize_window;
gboolean enable_java_applet;
+gboolean dns_prefetch;
};
#define WEBKIT_WEB_SETTINGS_GET_PRIVATE(obj)
(G_TYPE_INSTANCE_GET_PRIVATE((obj), WEBKIT_TYPE_WEB_SETTINGS,
WebKitWebSettingsPrivate))
@@ -157,7 +158,8 @@ enum {
PR
