On Sun, 24 Aug 2008, Eric Kidd wrote:
Greetings, Haskell folks!
I'm working on a toy program for drum synthesis. This has lead me to
explore the various sound-related libraries in Hackage. Along the way,
I've learned several things:
1. There's a lot of Haskell sound libraries, but no agreement on how
to represent buffers of audio data.
In my sound synthesis library
http://darcs.haskell.org/synthesizer/
I use mainly two representations:
1. Chunky sequences based on StorableVector. This is for efficient storage
and functions that need to process data in a non-causal way. They are not
good for fusion.
http://darcs.haskell.org/synthesizer/src/Synthesizer/Storable/Signal.hs
2. A Stream like sequences without wrappers. In fact, they are no data
storages but generator functions equipped with a seed. I use them for
causal signal processes. I think that most functions needed for drum
synthesis are causal. By inlining them, many subsequent causal signal
processes are fused to one.
http://darcs.haskell.org/synthesizer/src/Synthesizer/State/Signal.hs
2. Some of the most useful sound libraries aren't listed in Hackage's
"Sound" section, including HCodecs, SDL-mixer and hogg.
my library is also not at Hackage, it's too much experimental currently
3. The Haskell OpenAL bindings are great, but ALUT has been removed
from MacOS 10.5. Of course, a pure-Haskell version of ALUT would
be pretty easy to write, and it could be used as a fallback.
4. '[[a]]' is _not_ a good representation for multi-channel audio
data. Manipulating even 10 seconds of audio is noticeably slow.
Using the signal representations I mentioned above, I can synthesize some
songs in realtime. For stereo signals I use Signal (a,a). However, a
StereoFrame data type with Storable instance might be even better.
5. HCodecs is probably the best library for reading WAVE files. It
uses 'DiffUArray Int a' to represent audio data, and not '[[a]]'.
It also supports MIDI and SoundFont, which is quite handy.
6. YampaSynth looks really cool, and it might be possible to turn it
into a reusable MIDI synthesizer library for use with HCodecs.
This would be extremely useful: it would provide a basis for all
kinds of crazy MIDI-based programming in Haskell.
If you are after realtime synthesis you might also want to look into
SuperCollider. I posted some code to control SuperCollider via MIDI to
haskell-art. I can send it to you, if you like.
What would the ideal Haskell sound API look like? Personally, I would
love to see:
a. Something like HCodecs's Data.Audio as the basic sound format.
Data.Audio is an array-based API, and it supports converting
between a wide range of common sample formats. I don't know how
fast this is, but it already exists, and it's used by YampaSynth.
b. OpenAL for sound playback, with a portable version of ALUT. This
may require writing a pure-Haskell version of ALUT.
So far I used Sox' play command fed by a pipe with runInteractiveCommand.
Other nice-to-have features might include:
e. A standard MIDI format, based on either the HCodecs package or the
midi package. (HCodecs is used by YampaSynth, and the midi package
is used by alsa-midi.)
midi package is mainly used by Haskore
f. A modular version of YampaSynth which can convert MIDI data
structures into Data.Audio values.
It looks like Haskell could be a really sweet audio programming
environment with just a bit of integration work. What do folks think?
Are there other libraries I should look at more closely? Other features
that should be included in an ideal audio API?
What do you mean by 'API'? I think there should not be one library
handling all kind of music tasks. For interoperability common data
structures are nice. However, what signal representation concerns,
I think we need more than one representation, because different
applications have different needs.
Haskell audio libraries
I have advertised my libraries for years in Haskell-Wiki and HCAR, but it
seems that people do not look there. :-(
Reading and writing sound files:
HCodecs: (Audible a) => DiffUArray Int a
hsndfile: MArray with Float and Double
HSoundFile: [[Double]]
ALUT: OpenAL.AL.Buffer
WAVE: [[Int32]]
Playing sounds:
OpenAL: Ptr UInt8, Ptr Int16, 1-N channels
SDL-mixer: ForeignPtr ChunkStruct, ForeignPtr MusicStruct
Sox via pipe, using any data structure which can be written to a file
Sound processing libraries:
dsp: Array a, [a]
jack: Operates via mapping functions
mainMono :: (CFloat -> IO CFloat) -> IO ()
mainStereo :: ((CFloat, CFloat) -> IO (CFloat, CFloat)) -> IO ()
synthesizer: StorableVector, State, Generic (older: list, fusable list)
MIDI-based:
HCodecs: Reads and writes MIDI files
midi: Reads and writes MIDI files
alsa-midi: Uses midi library
YampaSynth: Stand-alone program
Haskore uses MIDI and other back-ends like Haskell Synthesizer, CSound,
SuperCollider
Special-purpose APIs only (FFIs, etc.):
hCsound: CsoundPtr
hsc3: UGen
HasSound
No public sound-buffer API:
hbeat: The relevant source files are missing!
hogg: Very low-level API for Ogg internals only
libmpd: No sound buffer API
sonic-visualizer: No sound buffer API
truelevel: Stand-alone program (uses WAVE)
wavconvert: Stand-alone program
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
