Re: [julia-users] Re: ANN: FileIO.jl, for loading and saving formatted files
Hello,
Picking up an old thread... I am looking for a way to trigger a specific
constructor when loading an object from a JLD file.
FileIO detects the files are JLD, and that is great, but it would be cool
if it could trigger my constructor `MyModule.MyType(objectfromjld)` when
- `MyModule` was specified in the addrequires while writing the file
- `JLD.exists(io, "MyType")` (namely the object `objectfromjld`)
(The case here is that MyType contains a field that is largish and that can
be precomputed from another field in the type, which is the want I want to
store in the file).
I've tried to write a `detectmytype()` in the spirit of, say,
`FileIO.detecthdf5()`, but I am kind-of stuck here:
- the detect...() interface wants an IO object (I think)
- I can find out if `MyType` exists from `jldopen(filename)`, but I can't
find an interface for hdf5/jld for an IO object.
Does anyone know how I could get this done?
Thanks,
---david
On Wednesday, September 30, 2015 at 11:35:20 AM UTC+2, Tim Holy wrote:
>
> Probably best as an issue over at FileIO.
>
> --Tim
>
> On Wednesday, September 30, 2015 01:24:25 AM David van Leeuwen wrote:
> > Hello Tim and Simon,
> >
> > I'd be very interested in this, for MFCCs.jl and perhaps
> > GaussianMixtures.jl. They currently use HDF5 and JLD, respectively, and
> I
> > don't think I've put in any magic.
> >
> > - is it wise to put in magic, even with container formats such as HDF5?
> >
> > - do you prefer questions about implementation on this list, the repo,
> or
> > some other way (or not at all)?
> >
> > Cheers,
> >
> > ---david
> >
> > On Tuesday, September 29, 2015 at 9:24:13 PM UTC+2, Tim Holy wrote:
> > > Simon Danisch and I are pleased to introduce FileIO.jl
> > > (https://github.com/JuliaIO/FileIO.jl), a package for centralizing
> the
> > > handling of formatted files. The package contains utilities for
> querying
> > > files/filenames by their extensions and/or magic bytes to deduce their
> > > content.
> > > It then allows package authors to "register" loaders/savers for
> detected
> > > file
> > > formats.
> > >
> > > Some of the advantages of using FileIO are:
> > >
> > > - For users, this may reduce the searching required to answer "is
> there a
> > > julia package to load file X?" Now you can just try `load("X.ext")`
> and
> > > see
> > > what happens---if the loader has been registered, it will try to find
> the
> > > right
> > > package for you.
> > >
> > > - For developers, this allows you to decouple your package from
> others. If
> > > you
> > > want to work with images, you will (once I tag a new release of
> Images.jl)
> > > no
> > > longer need to make the Images package a dependency as long as you use
> > > FileIO.
> > > In your package code, you can say `load("myimage.png")` and everything
> > > should
> > > Just Work.
> > >
> > > - For users and developers, this reduces name conflicts among two
> > > attractive
> > > function names, `load` and `save`. If everyone imports those names
> from
> > > FileIO, there need be no conflicts.
> > >
> > > The package is relatively small and is fairly extensively documented.
> > >
> > > While there are not a large number of formats currently registered,
> the
> > > architecture already shows a lot of promise. For example, it can
> recognize
> > > our
> > > .jld files as HDF5 files, something that escapes the well-tested (and
> much
> > > more
> > > extensive) Unix `file` command.
> > >
> > > Best,
> > > --Tim & Simon
>
>
Re: [julia-users] Re: ANN: FileIO.jl, for loading and saving formatted files
Probably best as an issue over at FileIO.
--Tim
On Wednesday, September 30, 2015 01:24:25 AM David van Leeuwen wrote:
> Hello Tim and Simon,
>
> I'd be very interested in this, for MFCCs.jl and perhaps
> GaussianMixtures.jl. They currently use HDF5 and JLD, respectively, and I
> don't think I've put in any magic.
>
> - is it wise to put in magic, even with container formats such as HDF5?
>
> - do you prefer questions about implementation on this list, the repo, or
> some other way (or not at all)?
>
> Cheers,
>
> ---david
>
> On Tuesday, September 29, 2015 at 9:24:13 PM UTC+2, Tim Holy wrote:
> > Simon Danisch and I are pleased to introduce FileIO.jl
> > (https://github.com/JuliaIO/FileIO.jl), a package for centralizing the
> > handling of formatted files. The package contains utilities for querying
> > files/filenames by their extensions and/or magic bytes to deduce their
> > content.
> > It then allows package authors to "register" loaders/savers for detected
> > file
> > formats.
> >
> > Some of the advantages of using FileIO are:
> >
> > - For users, this may reduce the searching required to answer "is there a
> > julia package to load file X?" Now you can just try `load("X.ext")` and
> > see
> > what happens---if the loader has been registered, it will try to find the
> > right
> > package for you.
> >
> > - For developers, this allows you to decouple your package from others. If
> > you
> > want to work with images, you will (once I tag a new release of Images.jl)
> > no
> > longer need to make the Images package a dependency as long as you use
> > FileIO.
> > In your package code, you can say `load("myimage.png")` and everything
> > should
> > Just Work.
> >
> > - For users and developers, this reduces name conflicts among two
> > attractive
> > function names, `load` and `save`. If everyone imports those names from
> > FileIO, there need be no conflicts.
> >
> > The package is relatively small and is fairly extensively documented.
> >
> > While there are not a large number of formats currently registered, the
> > architecture already shows a lot of promise. For example, it can recognize
> > our
> > .jld files as HDF5 files, something that escapes the well-tested (and much
> > more
> > extensive) Unix `file` command.
> >
> > Best,
> > --Tim & Simon
[julia-users] Re: ANN: FileIO.jl, for loading and saving formatted files
Hello Tim and Simon,
I'd be very interested in this, for MFCCs.jl and perhaps
GaussianMixtures.jl. They currently use HDF5 and JLD, respectively, and I
don't think I've put in any magic.
- is it wise to put in magic, even with container formats such as HDF5?
- do you prefer questions about implementation on this list, the repo, or
some other way (or not at all)?
Cheers,
---david
On Tuesday, September 29, 2015 at 9:24:13 PM UTC+2, Tim Holy wrote:
>
> Simon Danisch and I are pleased to introduce FileIO.jl
> (https://github.com/JuliaIO/FileIO.jl), a package for centralizing the
> handling of formatted files. The package contains utilities for querying
> files/filenames by their extensions and/or magic bytes to deduce their
> content.
> It then allows package authors to "register" loaders/savers for detected
> file
> formats.
>
> Some of the advantages of using FileIO are:
>
> - For users, this may reduce the searching required to answer "is there a
> julia package to load file X?" Now you can just try `load("X.ext")` and
> see
> what happens---if the loader has been registered, it will try to find the
> right
> package for you.
>
> - For developers, this allows you to decouple your package from others. If
> you
> want to work with images, you will (once I tag a new release of Images.jl)
> no
> longer need to make the Images package a dependency as long as you use
> FileIO.
> In your package code, you can say `load("myimage.png")` and everything
> should
> Just Work.
>
> - For users and developers, this reduces name conflicts among two
> attractive
> function names, `load` and `save`. If everyone imports those names from
> FileIO, there need be no conflicts.
>
> The package is relatively small and is fairly extensively documented.
>
> While there are not a large number of formats currently registered, the
> architecture already shows a lot of promise. For example, it can recognize
> our
> .jld files as HDF5 files, something that escapes the well-tested (and much
> more
> extensive) Unix `file` command.
>
> Best,
> --Tim & Simon
>
>
[julia-users] Re: ANN: FileIO.jl, for loading and saving formatted files
As Stefan - I love it. Imo, stuff like this really enhances the user
experience and productivity.
On Tuesday, September 29, 2015 at 3:24:13 PM UTC-4, Tim Holy wrote:
>
> Simon Danisch and I are pleased to introduce FileIO.jl
> (https://github.com/JuliaIO/FileIO.jl), a package for centralizing the
> handling of formatted files. The package contains utilities for querying
> files/filenames by their extensions and/or magic bytes to deduce their
> content.
> It then allows package authors to "register" loaders/savers for detected
> file
> formats.
>
> Some of the advantages of using FileIO are:
>
> - For users, this may reduce the searching required to answer "is there a
> julia package to load file X?" Now you can just try `load("X.ext")` and
> see
> what happens---if the loader has been registered, it will try to find the
> right
> package for you.
>
> - For developers, this allows you to decouple your package from others. If
> you
> want to work with images, you will (once I tag a new release of Images.jl)
> no
> longer need to make the Images package a dependency as long as you use
> FileIO.
> In your package code, you can say `load("myimage.png")` and everything
> should
> Just Work.
>
> - For users and developers, this reduces name conflicts among two
> attractive
> function names, `load` and `save`. If everyone imports those names from
> FileIO, there need be no conflicts.
>
> The package is relatively small and is fairly extensively documented.
>
> While there are not a large number of formats currently registered, the
> architecture already shows a lot of promise. For example, it can recognize
> our
> .jld files as HDF5 files, something that escapes the well-tested (and much
> more
> extensive) Unix `file` command.
>
> Best,
> --Tim & Simon
>
>
