On Thursday 11 September 2008 00:51, pmpp wrote: > Matthew Toseland a ?crit : > > On Thursday 11 September 2008 00:04, pmpp wrote: > >> Matthew Toseland a ?crit : > >>> On Wednesday 10 September 2008 18:42, pmpp wrote: > >>>> Matthew Toseland a ?crit : > >>>>> On Saturday 06 September 2008 23:21, pmpp wrote: > >>>>> > >>>>>> hi, i have made a small cifs experiment based on alfresco jlan-4.0 ( > >>> gpl ) > >>>>>> pure java on non windows , 2 jni dll on w32/64known problem: cifs IO > >>> will block on file copy/seeking till chk got at 100% > >>>>>> by freenet because no pseudo-randomaccessfile access to current > > download > >>> is > >>>>>> available with fcp. > >>>>>> > >>>>> Nice! Any filesystem interface is going to have the last problem. It's > > not > >>>>> something we can easily fix: we really don't want to fetch stuff from > > the > >>>>> beginning, because then the later parts will fall out first. A plugin > > for > >>>>> Explorer (etc) to show progress would be a possibility. > >>>>> > >>>>> ------------------------------------------------------------------------ > >>>> Thanks for you interest. > >>>> - imho plugin to show progress is not allowed at cifs protocol level > >>>> and is completely useless in multimedia appliance. > >>>> - You said "because then the later parts will fall out first " : > >>>> not sure , nowadays "files" are rarely fetch "from" the beginning . > >>>> especially with binary data container like mpeg > >>>> audio/avi/quicktime/matroska : these streams are indexed to allow > >>>> seeking. so application open a few headers blocks then jump straight to > >>>> the end of file to check bounds , and after it's a complex matter of > >>>> random access and prebuffering in the middle. > >>>> please note that in advanced video compression schemes, losing data ( > >>>> not keyframe ) in input stream is tolerated. > >>>> > >>>> -"easily fix" why fixing ? If some don't want data to be random > >>>> fetchable : why not precise it in metadata ? as this type of access > >>>> make sense only for indexed uncompressed streamable binary structure and > >>>> then allow another splitfileType (eg with compression off by default , > >>>> and FEC at user discretion , dedicated to random streaming of (volatile, > >>>> cause unpopular in 2 months ) multimedia crap ? , > >>>> I already did a raf experiment on <16meg chk, it was usefull to add > >>>> such a type because things won't broke outside the testing code/node. > >>>> > >>>> Freenet at first glance seems to be a distributed remote storage, it > >>>> would be nice to have some concepts of a real useable storage device in > >>>> it. even if i know its basicly only a distributed unreliable lru cache ( > >>>> I did same experiment in the past with chained squid and a samba plugin > >>>> after reading this > >>>> http://www.squid-cache.org/mail-archive/squid-dev/200002/0002.html ). > >>> Most of the time when somebody reads the file they will read it > > sequentially, > >>> because either: > >>> 1) They want to watch the movie/play the song from the beginning to the > > end, > >>> or > >>> 2) They want to copy it to another medium. > >>> > >>> IMHO different parts of a file falling out at different rates is > > unacceptable. > >>> The whole file should either survive or not. And of course for > > old/unpopular > >>> files, it may take a very long time to fetch the file: the download queue > > is > >>> the correct metaphor here, not the instantly accessible storage medium. > >>> > >>> > >>> ------------------------------------------------------------------------ > >> Sorry maybe misunderstanding, I didn't mean to modify the way download > >> queue work and then fetch data from freenet sequentially. I was just > >> thinking about to give an FCP option allowing client access to the block > >> currently pointed by a pseudo-randomaccessfile ( let say a 1mo buffer > >> forward bounding a long pointer ) as soon as possible when it's > >> available from node. That will be very appropriate especially when > >> nearly all buckets are already in cache/datastore. > > > > Files are divided up into 2MB segments. It might be possible to send each > > segment to the client when the segment is done; iirc I originally called this > > (not yet implemented) feature "ReturnType=chunked". It hasn't been > > implemented yet because there is no demand. However, we should still fetch > > blocks from a file in random order (although once we have a block succeeded > > in a segment, we tend to fetch the other blocks in that segment). > > > > Is this acceptable? > > > > > > ------------------------------------------------------------------------ > Well seems to be : on file <=2MB IO will not take more than 3 seconds > if buckets are in cache/store. But If file is really larger than 2mB > then it 's likely to be - very - usefull otherwise - even if buckets are > cached - client app could froze for minutes ... most of them don't > expect timeouts on cifs file and treat \\ unc as disk files ! > > "ReturnType=chunked" sounds good and maybe could give a starting point > for a jlan plugin ( CIFS/NFS/FTP bundle ) and even some kind of basic > http streaming :) > > note : for a good performance in media appliance, it may be usefull to > prefetch the last 2MB segment asap to fake to client that we have "all > file" present, I think most of mediaplayer don't really "read" the last > bytes of file, but most of them check end of file at opening.
Well, it's certainly possible to implement it. However, unless the file is in cache, any attempt at streaming a large file based on this is going to be *extremely* slow, unless we do non-random fetches, which I still think would be very iffy. IMHO you should look into providing a way to add stuff to the download queue, and then streaming stuff already downloaded. There are more advanced protocols than CIFS for this, such as what iTunes/mythtv use to talk to network peripherals... -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 827 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20080911/fd1d95fa/attachment.pgp>
