Here's my second proposal for GSoC, freshly popped into Melange. Thoughts,
comments and fish are all welcomed.
Proposal: Previewable Multimedia Fetches and Support for HTML5 Multimedia
Tags
Proposer: Spencer Jackson (sajack)
Introduction:
It would be very handy to support multimedia files on Freenet, especially
now that browsers are starting to support direct playback through HTML5.
However, these files are large, and there currently is no way for a user
tell if it is what he or she really wants until it's finished downloading. I
propose to implement a 'preview' mechanism which will allow a portion of a
file to be downloaded, so that it may be viewed and vetted. Below are the
steps I will take to implement this.
Implementation
First off, I will work on improving some of the content filters'
architecture, exposing a stream based API to the main application. This will
not take long, can easily integrate into the existing APIs, and could
improve performance in the future. As I'm going to be writing a new content
filter in a later step, it would be nice to get everything that needs
changing worked out of the underlying filtering system.
Right now, filters are a part of FProxy, and are invoked by it when a file
is downloaded. Really though, most clients probably desire filtered data, so
filtration should be done earlier, with FProxy simply using general
functionality. I will therefore move filters into the client layer, and
invoke them there. This must be done because many users will probably be
downloading content to their hard drives, and running it with a dedicated
media player. If we do not filter content stored on the hard drive, it will
all be unfiltered, and possibly dangerous when it is processed by
insufficiently paranoid applications.
Another thing while I'm working with filters in the client layer: I will
implement filtration of inserts. This will help prevent metadata in a file
uploaded by the user from breaking his or her anonymity. For example, EXIF
data in jpegs may reveal the serial number of the camera which took the
picture, or even the GPS coordinates from where the picture was taken. This
shouldn't be too difficult to implement, and will produce a pleasing
symmetry in the next step.
There are some usage cases, such as during debugging, where it may be
undesirable for a request to be filtered. It must therefore be possible to
disable filtering. To accomplish this, I will prevent the data from being
filtered when a configuration setting in the request's context has been set.
Support for disabling filters will need to be added to FCP. All of this will
then need to be supported in the web interface. I will add support for
complementary GET and POST variables in FProxy which would be used to
trigger this setting. Next, I'll add UI elements to the download and insert
queue pages and any other pertinent locations, such as the 'Downloading a
page' page, which would enable these variables. These elements should only
be visible when the user is in 'Advanced mode,' and, even then, should be
tagged with a Big Fat Warning about the risks of turning off filtering.
Next, I will write an Ogg container filter. This will be technically
interesting, as this format encapsulates other types of data. A generic Ogg
parser will be written, which will need to validate the Ogg container,
identify the bitstreams it contains, identify the codec used inside these
bitstreams, and process the streams using a second(or nth, really, depending
on how many bitstreams are in the container) codec specific filter. It
should be possible to use this filter to either filter the just
beginning(the first segment) of the file, or the whole thing. This is what
will make it possible to preview a partially downloaded file. Some things
which will need to be taken into consideration are the possibility of Ogg
pages being concealed inside of other Ogg pages. This will be checked for,
and a fatal error will be raised if it occurs. Two codec specific filters
which are of high priority are Vorbis and Theora, as these are commonly used
in HTML5 video. After this is complete, the HTML content filter will need to
be modified to allow HTML5 multimedia specific tags which may contain this
data.
I will then modify fetching to make it possible to download the first
segment of a multimedia file before any others. This segment will probably
contain the headers of the file, along with some content. I will further
modify the process of fetching so that it is possible to retrieve this
segment before the rest of the file has finished downloading. This shall be
immediately exposed via FCP.
Support for retrieving the first segment should then be added to FProxy.
When downloading a file, if the first segment has been downloaded, but the
file has not been completely retrieved, the first segment should be filtered
and returned to the browser, which will probably be rendering it through an
HTML5 multimedia element. If the entire file has downloaded, that should be
filtered and returned instead.
The next step is to make this more intuitive for the end user, who might be
wondering, "Why do I only see a part of this file? Why do files sometimes,
but not always, suddenly play when I refresh?" The solution is to add UI
elements from the web-pushing branch, which has systems to inform the
browser when an image embedded in a freesite has been fetched, and should be
displayed, with no refreshing required. It can even show progress bars. I
will extend these systems to apply to multimedia files. Furthermore,
Javascript will be used to enhance the browser's native player. While
attempting to preview a file, the first segment will be downloaded, as per
the previous steps. When this has been retrieved, the browser will be
notified via web-pushing, and it will retrieve the filtered segment and
display it, via its support for HTML5. The fetch for the rest of the file
will continue in the background. When it has completed, web-pushing will be
used to notify the browser, and the multimedia element will again be
updated. During this process, web-pushing will be used to render progress
bars and inform the user of changes in the fetch's status. A UI element in
the player will be click-able to add the file to the download queue page.
Milestones
*Stream based filters (4 days)
*Filters are moved to the client layer, with support for (disableable)
support filtering files going to the hard drive, and inserts (10 days)
*An Ogg content filter is implemented, supporting the following codecs: (4
days)
-The Vorbis codec (3 days)
-The Theora codec (3 days)
*First segments of multimedia files are downloaded first and are retrievable
by clients through FCP (8 days)
*FProxy will returns the filtered first segment of a multimedia file, if it
is available, and the whole file has not been fetched (3 days)
*A 'preview' button is available on FProxy's download queue page (2 days)
*FProxy uses web-pushing to, from an HTML5 element, report on progress of a
fetch, and reload a multimedia element when more has become available for
rendering (10 days)
Biography
I initially became interested in Freenet because I am something of a
cypherpunk, in that I believe the ability to hold pseudonymous discourse to
be a major cornerstone of free speech and the free flow of information. I've
skulked around Freenet occasionally, even helping pre-alpha test version
0.7. But I'd like to do more. I want to put my time and energy where my
mouth is and spend my summer making the world, in some small way, safer for
freedom.
Starry-eyed idealism aside, I am an 18 year old American high school senior,
who will be studying Computer Science after I graduate. While C/C++ is my
'first language', so to speak, I am also fluent in Java and Python. Last
year, I personally rewrote my high school's web page in Python and Django.
This year, I've been working on an editor for Model United Nations
resolutions, as time permits. This project is licensed under the GPLv3, and
is available on GitHub, at http://github.com/spencerjackson/resolute. It's
written in C++, and uses GTKmm for the GUI.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://emu.freenetproject.org/pipermail/devl/attachments/20100408/5664eebe/attachment.html>
