Re: [XSCE] Re: [Server-devel] ext2 vs ext4 vs exFAT for XO content SD cards?

2015-08-16 Thread Anish Mangal
Just to chime in, as has been already noted the max file size on a FAT32
system is 4GB. Some of the files we deal with are much larger than that.
Ex. the Zim files for TED talks etc. are 8GB+ in size. Now we could always
break them into smaller chunks, but that is another step.

--
Anish


On Mon, Aug 17, 2015 at 9:44 AM, James Cameron  wrote:

> On Sun, Aug 16, 2015 at 11:24:23PM -0400, Adam Holt wrote:
> > On balance, SD Card industry standard exFAT seems (to me) more
> > future-proof for a hassle-free "grassroots content" partition over
> > coming years,
> > [...]
>
> If you're able to control the desktops and laptops that will be used
> to add or remove content, sure, but exFAT is quite recent as far as
> remote villages are concerned; what will you do when you get teachers
> who can't even open it?  Any systems with unpatched XP or Vista will
> be affected.  Any Mac earlier than 10.6.5 too.
>
> --
> James Cameron
> http://quozl.linux.org.au/
>
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel


Re: [XSCE] Re: [Server-devel] ext2 vs ext4 vs exFAT for XO content SD cards?

2015-08-16 Thread James Cameron
On Sun, Aug 16, 2015 at 11:24:23PM -0400, Adam Holt wrote:
> On balance, SD Card industry standard exFAT seems (to me) more
> future-proof for a hassle-free "grassroots content" partition over
> coming years,
> [...]

If you're able to control the desktops and laptops that will be used
to add or remove content, sure, but exFAT is quite recent as far as
remote villages are concerned; what will you do when you get teachers
who can't even open it?  Any systems with unpatched XP or Vista will
be affected.  Any Mac earlier than 10.6.5 too.

-- 
James Cameron
http://quozl.linux.org.au/
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel


Re: [XSCE] Re: [Server-devel] ext2 vs ext4 vs exFAT for XO content SD cards?

2015-08-16 Thread Adam Holt
On Sun, Aug 16, 2015 at 10:57 PM, James Cameron  wrote:

> LFN (long filename) support is present in all the operating systems
> you've mentioned, and works fine with FAT32.
>

Thanks for the correction.  FAT32 is indeed about as universal/tolerable of
a standard as possible in 2015.  But might not be wise on large SD cards
going forward, given 32GB is the traditional Windows limit for FAT32
partitions.  And of course FAT32's a bit slower than modern filesystems,
and worse 4GB content zip/repos (FAT32's upper limit) will appear before we
know it?

On balance, SD Card industry standard exFAT seems (to me) more future-proof
for a hassle-free "grassroots content" partition over coming years, for 1%
of the SD card of whatev.  Or perhaps NTFS ~if~ file owners/permissions are
more important than Mac support.

In the OLPC ecosystem, only Open Firmware has an 8.3 limit, and there's
> nothing in your use case which suggests that Open Firmware is to be
> used for content delivery!
>

:)


-- 
Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel


Re: [XSCE] Re: [Server-devel] ext2 vs ext4 vs exFAT for XO content SD cards?

2015-08-16 Thread James Cameron
On Sun, Aug 16, 2015 at 09:12:34PM -0400, Adam Holt wrote:
> Towards this quite universal demand, an exFAT partition seems much
> better than FAT32, as exFAT works with most all recent Windows and
> Mac machines, without filename limitations.  (Not unrelated to exFAT
> being the modern SD Card industry standard.)  Reason being that
> teachers/principals/pedagogy folk just don't have hours and hours to
> squash every filename down to FAT32's 8+3 character limit.

You are quite wrong here.

LFN (long filename) support is present in all the operating systems
you've mentioned, and works fine with FAT32.

In the OLPC ecosystem, only Open Firmware has an 8.3 limit, and there's
nothing in your use case which suggests that Open Firmware is to be
used for content delivery!

-- 
James Cameron
http://quozl.linux.org.au/
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel


Re: [XSCE] Re: [Server-devel] ext2 vs ext4 vs exFAT for XO content SD cards?

2015-08-16 Thread Adam Holt
A "Custom_Content" folder is eventually demanded by almost every IIAB-like
deployment.

The reason is that every local school / librarian quite naturally wants a
Non-Bureaucratic process, to add their own language/videos/curriculum,
copying their own file-tree onto the SD card, using their own Windows/Mac
computer hassle-free.

Towards this quite universal demand, an exFAT partition seems much better
than FAT32, as exFAT works with most all recent Windows and Mac machines,
without filename limitations.  (Not unrelated to exFAT being the modern SD
Card industry standard.)  Reason being that teachers/principals/pedagogy
folk just don't have hours and hours to squash every filename down to
FAT32's 8+3 character limit.

Likewise for the same reason NTFS is another strong possibility, with
better file ownership/permissions possibilities than exFAT, however NTFS
does not work out-of-the-box on Mac.  Which doesn't much matter across the
preponderance of lower-class schools planetwide, where Windows 7 and
similar are the norm.  However deep-pocketed NGO educators bring
Mercedes-class MacBooks to the table for sure; yes rich voluntourists can
be annoying but when they invest for the long-term let's absolutely make a
place for them too =)


On Sun, Aug 16, 2015 at 8:31 PM, James Cameron  wrote:

> No, it won't help.
>
> 1.  reading data an SD card (or eMMC, or USB flash drives) does cause
> writes internal to the card, and does reduce life,
>
> 2.  there's no such flag to set in an MBR partition table,
>
> Perhaps you mean a write protect switch on the card?  This is a
> plastic slide, sensed by a switch in the socket, and provides _advice_
> to the system.  Some systems don't receive the advice.  The system can
> ignore that advice.  Even if acted upon, it only stops change to the
> data, it doesn't stop writes internal to the firmware of the card.
>
> On Sun, Aug 16, 2015 at 05:18:01PM -0700, Sameer Verma wrote:
> > James,
> > Would it help to mark the content partition(s) as read only?
> >
> > Sameer
> >
> > On Aug 16, 2015 5:13 PM, "James Cameron" <[1]qu...@laptop.org> wrote:
> >
> > Thanks, interesting questions.
> >
> > No, ext4 is not a slow journaled filesystem, and no, there are no
> > obvious problems on SD when using ext4 given your use case.  But it
> > isn't operating system portable, and as your content is static no
> need
> > for a journal.  Other features of ext4 make mounting or filesystem
> > check faster.
> >
> > Yes, wear-leveling is taken care of by the firmware in the card, put
> > there by the manufacturers.  Wear-levelling also critical during
> > reading, since a flash page can't be read repeatedly without
> > disturbance eventually requiring it to be written to a freshly erased
> > page.  This is all handled by the firmware.  Happens way more
> > frequently than it does on a hard drive.
> >
> > Duplication time of SD cards won't be affected by your filesystem or
> > partition decision.
> >
> > One partition is sufficient.  MBR partition type best, for
> > compatibility across the operating systems.
> >
> > For filesystem, use FAT32, mounted read-only.  FAT32 works across
> most
> > Windows and Mac computers, at media sizes up to 2 TB, for file sizes
> > up to 4 GB.
> >
> > Where content cannot live on FAT32 due to file name character set or
> > metadata, it can be placed in disk image bundles of ISO-9660,
> > squashfs, or ext4 and loop mounted.  The content curation process for
> > the end user might be easier if bundles can be added and removed as
> > needed.
> >
> > What you might be overlooking; I/O bandwidth of the connection to the
> > media, endurance impact of reading data from the card slowing
> > performance one year on, backups, content bundle tamper checks, risk
> > of filesystem format incompatibilities introduced by new versions of
> > operating systems after your product is in the field, risk of cross
> > system malware infections, electrostatic discharge damage to the
> card,
> > and how modern cards can change performance behaviour as a result of
> a
> > production state awareness flag stored by card firmware.
> >
> > On the other hand the alternatives have their own problems.
> >
> > --
> > James Cameron
> > [2]http://quozl.linux.org.au/
> > ___
> > Server-devel mailing list
> > [3]server-de...@lists.laptop.org
> > [4]http://lists.laptop.org/listinfo/server-devel
> >
> > References:
> >
> > [1] mailto:qu...@laptop.org
> > [2] http://quozl.linux.org.au/
> > [3] mailto:server-de...@lists.laptop.org
> > [4] http://lists.laptop.org/listinfo/server-devel
>
> > ___
> > Devel mailing list
> > Devel@lists.laptop.org
> > http://lists.laptop.org/listinfo/devel
>
>
> --
> James Cameron
> http://quozl.linux.org.au/
>
> --
>