I have merged the web-pushing branch. It is configurable, and disabled by default. This is the result of sashee's Summer of Code project last year. It uses GWT to update HTML on the fly. Initially this is the progress bars, and inline images. web-pushing has serious known bugs and is likely to remain off by default for some time, but maintaining it out of tree was rather costly because it affects all the web interface classes - there is code to push various generated web pages which was removed and can now be put back in.
To do a full rebuild (ant distclean; ant), you will need the GWT jars. However the generated javascript is checked in to version control for convenience. Brief changelog: - The web-pushing branch introduces web pushing to the Freenet user interface. That is, various elements of the user interface - alerts, progress bars, some generated pages - are pushed dynamically whenever the underlying data changes. - The client side is javascript, but this javascript is generated by GWT from Java source in generator/js/src/. GWT is used in a very low-level way here, essentially just to replace tags (so the base HTML is still there, and it works fine with javascript turned off, it just doesn't push), but it is efficient, we share a single connection between any number of browser windows. - To fully rebuild fred including the javascript, thus, you need the GWT jars, gwt-dev.jar and gwt-user.jar, in generator/js/lib/. This is deleted if you do ant distclean but is checked in to version control for convenience. - One important complication is that it is very hard to do a fully clean build of GWT - the build relies on many third party jars bundled in SVN. The easiest option is just to drop Google's jars in. Help with automating this would be appreciated. README.clean-build explains this and similar (but less major) issues with freenet-ext.jar. - UpdaterConstants.java is shared between the client and server layers. It is copied during the build process. - The server side is normally notified when a window is closed. If it doesn't get the callback there is a timeout: We send keepalives every 10 minutes and if one is not received in 21 minutes we timeout. - If debugging is enabled (compile time option), the client side logs to the server side and the logs end up in logs/. - All these notifications are handled through new toadlets. - There are many different types of tag replacements supported by the client. These are used for different purposes. E.g. on some pages we need to be able to change the page title as well as elements on the page. Some are just countdowns, implemented on the client side. Etc. - Pushing is implemented by "long polling", i.e. we send a request and get a response when something happens. If the client fails to connect it does exponential backoff but only up to 10 seconds delay between attempts. - One window is the "leader", which actually connects to the server side (fproxy), and fetches the updates. All notifications that it doesn't use locally it forwards to other windows via cookies. - Followers check for notifications every 100ms. - There is support for showing messages at the top of a page e.g. notifications of errors (e.g. backend has gone away), the page has been updated etc. Currently this is only used for error reporting and showing the overall status on a page with lots of images. In future we may show more messages; the code supports showing useralerts but this is disabled. - Polling just returns element ID's. We then request the new data for that element, and apply it according to the tag replacement type. - An important part of the functionality is the image loading support. On a page like The Activelink Index with lots of inline images, we show 1) a generated image showing the percentage completion of each image, and 2) a progress bar at the top giving overall progress in number of blocks. This not only looks better but it also eliminates the many-connections bug - browsers generally only allow 6 or 8 connections, so fetching 6 at a time is very slow on Freenet and blocks any other Freenet browsing. - All this requires localisation. L10n messages for the client are merged into fred's l10n and passed to the client. - generator/js/tests/src/freenet/PushTester.java is a set of client-level tests. Not sure how to invoke it, maybe from the push tester page? - Changes to fproxy fetch listener architecture, implementation changes, some relatively minor refactoring. - FProxyFetchListener: Gets notified of every change. Used by web-pushing to push status updates on progress bars. - Fproxy progress-page fetches can be cancelled. This happens when we get a callback because the page has been closed. - FProxyFetchTracker is passed in to fproxy, not created by it. - Image loading code requires we modify the HTML (and disallow serving content as XHTML, as GWT doesn't work with it). We achieve this by PushingTagReplacerCallback, which uses the new TagReplacerCallback infrastructure. - In many places, HTML is factored out to an Element, which generates the HTML, but which can be called on the fly when it needs to be updated. So we just add the Element. If web-pushing is enabled, it will be generated and then updated on the fly; if it isn't, it will just be generated once. - Check for whether a FProxyFetchInProgress has already fetched the data, in the blocking path in fproxy. Race condition. - Image progress is constructed by a toadlet that makes a PNG containing the given text. This seems to be easier than the alternative. - Some elements need to be refreshed every 10 seconds. IntervalPusherManager deals with this. - Add the jsonly class in PageMaker for stuff that should be rendered if javascript is enabled. Send the actual GWT-generated javascript in /static/freenetjs/freenetjs.nocache.js; this is <script>'ed both in fproxy (for image updating etc) and in PageMaker. - Send a requestId and the client-side L10n in PageMaker. - PushDataManager handles all pushing, is created by SimpleToadletServer. - ToadletContainer etc: isFProxyWebPushingEnabled(). - ToadletContext: getUniqueId() (= request ID for the page). - Add code for parsing HTTP dates in ToadletContextImpl, used by image generator toadlet. - DismissAlertToadlet: Disable a toadlet from the user clicking on the alert on the client side. Disabled at the moment. - LogWritebackToadlet: Write logs from client side for debugging. - PushDataToadlet: Get the data for an updated element (base64'ed). - PushFailoverToadlet: Used to failover from one leader to another. - PushKeepaliveToadlet: Used to send keepalives. - PushLeavingToadlet: Used to tell the node that a page was closed, and all its inline images are no longer needed, or the page itself isn't needed if it is a progress page. - PushNotificationToadlet: Server side of long pulling. Blocks until there is a notification and returns its ID. - PushTesterToadlet: Renders a page full of tester elements. - SecondCounterNode: A second counter (up or down, either way implemented on client side). - ContentFilter: HTML_MIME_TYPES: Here so we can use it for identifying HTML elsewhere. - ContentFilter: Take TagReplacerCallback callback so can modify tags. Make HTMLFilter.ParsedTag public so that TagReplacerCallback can modify them. Arguably this should happen at a different level, as ParsedTag is HTML-specific. - FilterCallback: processTag() for tag replacements - normally via a TagReplacerCallback, GenericFilterCallback chains to one. - HTMLFilter: Ensure there is a <head>. We need to add stuff to it for image pushing. - Some extra methods/constructors for ParsedTag to make modifying them easier. - PushingTagReplacerCallback: Replace <img>'s with ImageElement's so that we can fetch them all at once on the server side and push updates to indicate progress; insert request ID and client-l10n data (and the in-page alerts code, but currently disabled); and insert the generated javascript. - URIProcessor: Utility interface implemented by GenericReadFilterCallback to help PushingTagReplacerCallback. - Minor fix in old progress page javascript. - Customised version of Reset CSS (public domain) used for alerts-on-top-of-page pushing (currently disabled). - BaseUpdateableElement: Basis of all pushed HTMLNode's. - ImageElement: Implementation of image progress pushing. Follows permanent redirects automatically (handy with USK activelinks). Tracks an FProxyFetchInProgress. Pushes progress image, initially "Initializing" and later X%; also pushes hidden fields with block counts for the overall block count displayed as a top-message/overlay alert. - NotifierFetchListener: Connect an FProxyFetchListener to a BaseUpdateableElement - update the element whenever the listener makes progress. - ProgressBarElement: A progress bar based on an FProxyFetchWaiter. Uses NotifierFetchListener. If finished, failed, or has data, reload the page. Otherwise render a progress bar, in much the same way as we did before. This does not include progress info - MIME type etc. - ProgressInfoElement: Renders the rest of the information on a progress page. Filename, MIME type, block counts in advanced mode, time elapsed, ETA (the last two as SecondCounterNode's), text explaining current progress. - PushDataManager: Tracks pushed elements, notifications waiting, checks for timeouts, etc. - TesterElement: Second counter counts on server side. Only for testing pushing. - PeerManager: Add a list of PeerStatusChangeListener's to be notified when peers' status codes change. This can be used to push peer status changes. (But currently that code is removed to make merging easier). We also register the listeners on each PeerNode, which uses a WeakHashSet so we only need to deregister on PeerManager. PeerNode also sends notifications. - Whiteboard: In-memory global download queue notifications system, used to send updates whenever the queue page changes. Again the actual queue page pushing is not enabled in the current code. But the global clients are hooked in to Whiteboard. - UserEventListener: Infrastructure for pushing useralerts. Again this is disabeld for now. - HTMLNode: Allow content in more cases. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100426/5820b37e/attachment.pgp>
