Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
great! I am wondering if you can provide even higher-level APIs for the common case: hash - runResourceT $ hashFile my-file and possibly something that runs the ResourceT transformer: hash - runHashFile my-file On Sat, Jan 7, 2012 at 12:16 AM, Felipe Almeida Lessa felipe.le...@gmail.com wrote: Hello! I'm pleased to announce the first release of crypto-conduit [1]! The crypto-api [2] package provides APIs for many cryptographic operations, such as cryptographic hashes and block ciphers. This new crypto-conduit package allows you to use many of these operations with conduits [3], giving you safe I/O using constant memory and no leaks. As an example, here's how you could get the SHA1 hash a file: import Crypto.Conduit -- from crypto-conduit import Crypto.Hash.SHA1 (SHA1) -- from cryptohash import Data.Conduit -- from conduit import Data.Conduit.Binary (sourceFile) -- from conduit main = do hash - runResourceT $ sourceFile my-file $$ sinkHash print (hash :: SHA1) The code snippet above, despite having only sourceFile ... $$ sinkHash on its core, guarantees that the file handle is not kept open and uses a constant amount of memory. Sweet! Please break this package! Although it comes with a test suite, it has just seen the light of the day. Cheers, =) [1] http://hackage.haskell.org/package/crypto-conduit [2] http://hackage.haskell.org/package/crypto-api [3] http://hackage.haskell.org/package/conduit -- Felipe. ___ web-devel mailing list web-de...@haskell.org http://www.haskell.org/mailman/listinfo/web-devel ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
On Sat, Jan 7, 2012 at 8:06 AM, Greg Weber g...@gregweber.info wrote: I am wondering if you can provide even higher-level APIs for the common case: hash - runResourceT $ hashFile my-file and possibly something that runs the ResourceT transformer: hash - runHashFile my-file That's dead simple to add, I just wonder which ones should be added (since triplicating the whole API wouldn't be fun). So you're assuming that hashing is the most common case of the library, right? Now, having 'hashFile' inside ResourceT isn't terribly useful and if the user needs it, it's trivial to implement, so I'm thinking of just exporting a single new function: hashFile :: (MonadIO m, Hash ctx d) = FilePath - m d I'll include it on the next version. =) Cheers, -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
And while we're at it, some code to deal with the cumbersome decoding of those hash objects would be nice! Cheers, Aristid Am 07.01.2012 11:07 schrieb Greg Weber g...@gregweber.info: great! I am wondering if you can provide even higher-level APIs for the common case: hash - runResourceT $ hashFile my-file and possibly something that runs the ResourceT transformer: hash - runHashFile my-file On Sat, Jan 7, 2012 at 12:16 AM, Felipe Almeida Lessa felipe.le...@gmail.com wrote: Hello! I'm pleased to announce the first release of crypto-conduit [1]! The crypto-api [2] package provides APIs for many cryptographic operations, such as cryptographic hashes and block ciphers. This new crypto-conduit package allows you to use many of these operations with conduits [3], giving you safe I/O using constant memory and no leaks. As an example, here's how you could get the SHA1 hash a file: import Crypto.Conduit -- from crypto-conduit import Crypto.Hash.SHA1 (SHA1) -- from cryptohash import Data.Conduit -- from conduit import Data.Conduit.Binary (sourceFile) -- from conduit main = do hash - runResourceT $ sourceFile my-file $$ sinkHash print (hash :: SHA1) The code snippet above, despite having only sourceFile ... $$ sinkHash on its core, guarantees that the file handle is not kept open and uses a constant amount of memory. Sweet! Please break this package! Although it comes with a test suite, it has just seen the light of the day. Cheers, =) [1] http://hackage.haskell.org/package/crypto-conduit [2] http://hackage.haskell.org/package/crypto-api [3] http://hackage.haskell.org/package/conduit -- Felipe. ___ web-devel mailing list web-de...@haskell.org http://www.haskell.org/mailman/listinfo/web-devel ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
On Sat, Jan 7, 2012 at 9:12 AM, Aristid Breitkreuz arist...@googlemail.com wrote: And while we're at it, some code to deal with the cumbersome decoding of those hash objects would be nice! I'm sorry, but what do you mean by cumbersome decoding? Cheers, =) -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Motivation for having indexed access in Data.Map?
Hello, I wonder why Data.Map provides the indexed access functions: http://hackage.haskell.org/packages/archive/containers/latest/doc/html/Data-Map.html#g:21 These functions seem rather out-of-place to me in the map api. The only use case I could think of so far would be to find the median, or in general n-th smallest key, but that does not seem sufficient reason (also, I think there are faster methods for that). Anything else? Regards, Chris ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Problems with libgmp on Amazon Linux
Am 07.01.2012 06:04, schrieb Yucheng Zhang: On Sat, Jan 7, 2012 at 8:08 AM, Nicu Ionitanicu.ion...@acons.at wrote: It seemed to work pretty well, with some problems when configuring HP, because although libgmp was already installed, it was not recognized. I installed the newer version 5.0.2 (from sources) [4], did some tricks (cp /usr/lib/lib/libgmp* /usr/lib), then configure worked and HP was built and installed. I think you should get the installed libgmp recognized, but not doing tricks like this. It is no problem as a temporary solution, but you should always try to make use of a package manager. Loading package integer-gmp ...command line: can't load .so/.DLL for: gmp (libgmp.so: cannot open shared object file: No such file or directory) I have no experience with AWS, but it seems that you need to run a ldconfig to rebuild the cache, since you installed libgmp just by copying into /usr/lib. ldconfig -n /usr/bin Thank you, a good point. Just to let a trace for the posterity, here is what I did: I now deleted libgmp* from /usr/lib (which I copied from /usr/lib/lib, where gmp did intstall, although I configured it with --prefix=/usr/lib), then ran: sudo ldconfig /usr/lib/lib which updated also the ld cache, and then I defined (in .bash_profile) LD_LIBRARY_PATH=/usr/lib/lib After a new login everything could be installed (I installed aws and criterion, after which half of Hackage was on my machine ;-) This installation point of gmp is a bit odd, but I don't want now to find out why the gmp intallation procedure doesn't honor --prefix. After this I made an AMI (image) of it, which could be used as a start point for further installations. I did not test the installation extesively yet, but by so many succefully installed packages, I'm confident that everything is ok. Next days I'll play with it. By the way, I let gmp 5.0.2 there, it looks that HP 2011.04 likes it. Nicu ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Motivation for having indexed access in Data.Map?
On 07/01/2012 12:17, Christoph Breitkopf wrote: Hello, I wonder why Data.Map provides the indexed access functions: http://hackage.haskell.org/packages/archive/containers/latest/doc/html/Data-Map.html#g:21 These functions seem rather out-of-place to me in the map api. The only use case I could think of so far would be to find the median, or in general n-th smallest key, but that does not seem sufficient reason (also, I think there are faster methods for that). Anything else? I don't know the motivation in Data.Map, but here's some thoughts from a C++ home-rolled data structures perspective... Somewhere around a decade ago, I started an in-memory C++ multiway tree library, initially an experiment seeing if I could improve sequential access performance. This half-worked, but I still use the data structure primarily because it's a bit safer in some cases than the STL containers, and also has some extra functionality that makes it more convenient. Features include... 1. cursor maintenance (when I insert/delete, cursors/iterators are not invalidated except in the special case that the cursor references an item that is deleted. There are two tricks for this case - the cursor will at least know that the item is deleted, plus there are special cursors that can defer deletion (mainly for delete-the-current-item within loops). 2. Searching based on custom comparisons - mainly searching based on a partial key (certain fields), so you can find the first/last item equal to a partial key, ignoring less significant fields. 3. Finding the first key that is *not* in the container (for unsigned integer keys only). 4. Subscripted access - finding a given index, determining the index to an item referenced by a cursor, stepping forward/backward by a given number of items. The subscripted access isn't massively useful - it was implemented because I was curious how to handle it efficiently. However, cases do come up from time to time in strange places. For example, sometimes it's more convenient to store an index (into a container that won't change) than a cursor or a full key. And using an ordered container does tend to imply, after all, that you're interested somehow in the order (or else why not use a hash table?). One case, I guess, relates to DSL-generated data structures. The point there is that when the generated code runs, the map instance is long dead. Within the generated code, ranges etc tend to be identified by subscript - so the DSL needs to be able to translate from key to subscript, and (maybe) back again. OTOH, don't forget that laziness thing - if the code generator was working from a sorted array it would know the subscripts anyway. A particularly surprising side-effect - along with the map, multimap, set and multiset wrappers, I have a vector wrapper. When you have a huge array and do lots of inserts and deletes within that array, a multiway tree (with subscripted access) turns out to be a good trade-off. Some accesses are more awkward (because the items aren't all contiguous in memory), but the log-time inserts and deletes can be worth it. The first-key-not-in-the-container stuff was mostly a side-effect of the data structure augmentation I did for subscripted access. That is very convenient, but with costs. The no. 1 killer feature that keeps me using and maintaining this library is the partial-key search thing. This is so useful, I even added a feature to a DSL (used mainly for AST nodes and multiple dispatch - originally based on treecc) to make it more convenient to generate partial-key classes. The cursor maintenance makes it a lot easier to write algorithms that update the container, but it's perhaps surprising how rare that's necessary. The issue with all this is of course partly overhead, but also because I got lazy - keeping these things hanging around throughout whole program runs like cheap second-rate databases. They are quite convenient to work with, but for a long time I stopped even considering pulling all the data out into a flat array, processing it there, then rebuilding a new indexed data structure only if I really needed it, or keeping data mostly in an array and sorting it ready for binary searches just at the key point where that's needed. Some programs I've written using them are maybe an order of magnitude slower than they should be, and in quite a few cases there's an asymptotic difference, not just a constant factor - a lot of algorithms are O(n log n) where without the convenience containers they could be O(n). Very little of this would be relevant in a pure functional programming world, of course, but anyway - yes, subscripting can be (occasionally) useful. It's just hard to give specific examples, because they're buried in all the technicalities of quite large programs. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org
[Haskell-cafe] ANNOUNCE: DHD = UHac, April 20 - 22, Utrecht
Calling all Haskellers! To open 2012 and in celebration of the group's 3rd year of existence, the Dutch Haskell Users Group is happy to announce a new event that combines an informal conference and a hackathon. = = = = =*DHD = UHac* = = = = *April 20 - 22* = = = = *Utrecht* = = = = = = DHD = The Dutch HUG Day, or DHD, will be a half day of talks on anything Haskell-y or FP-ish. It will be held on Friday, April 20. = UHac = The Utrecht Hackathon, or UHac, will be a gathering of the community of enthusiastic Haskell hackers from around the world. It will start immediately following the DHD and continue to Sunday, April 22. = Call for Speakers = We are looking for speakers for the Dutch HUG Day. If you have something interesting to share, please email Sean Leather (leat...@cs.uu.nl). See the website for more. = Website = For more information that will be continuously updated, see the event wiki page: *http://www.haskell.org/haskellwiki/DHD_UHac* Join us for a Haskell-y time in the Netherlands. We look forward to seeing you there! Sean Leather and Jurriën Stutterheim Organizers ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
Well, how do you get a ByteString from the hash object? Aristid Am 07.01.2012 13:04 schrieb Felipe Almeida Lessa felipe.le...@gmail.com: On Sat, Jan 7, 2012 at 9:12 AM, Aristid Breitkreuz arist...@googlemail.com wrote: And while we're at it, some code to deal with the cumbersome decoding of those hash objects would be nice! I'm sorry, but what do you mean by cumbersome decoding? Cheers, =) -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
On Sat, Jan 7, 2012 at 12:16 PM, Aristid Breitkreuz arist...@googlemail.com wrote: Well, how do you get a ByteString from the hash object? Just use encode from Data.Serialize. =) Cheers, -- Felipe. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
Yeah and that's annoying IMHO. :) It's not really important though. Aristid Am 07.01.2012 15:39 schrieb Felipe Almeida Lessa felipe.le...@gmail.com: ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OpenAL bindings?
Jason Dagit wrote: Looks like the repo [1] for the OpenAL bindings that Sven Panne created [2] is no longer available. I assume this is a result of The Great Server Outage of 2011 [3]. [1] http://darcs.haskell.org/packages/OpenAL/ [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/OpenAL [3] http://www.haskell.org/pipermail/haskell-cafe/2011-February/088829.html No, darcs.haskell.org is not the community server. It is a separate server that seems to be located at Galois. I see that the following darcs repositories are available on the community server: http://code.haskell.org/OpenAL/OpenAL http://code.haskell.org/OpenAL/ALUT Hope this helps, Yitz ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANN: wxHaskell 0.13.2
Hi guys, I am please to announce that wxHaskell 0.13.2 has just been uploaded to Hackage. when I try to build the latest version on Linux/x86_64 running NixOS, I get the following error at configure time: Setup: Missing dependency on a foreign library: * Missing C library: wx_gtk2u_media-2.8 I searched my hard disk for that library, and apparently my installed copy of wxGTK-2.8.12 doesn't have it. There are plenty of libwx_gtk2u_* libraries, but none of them is called media. Does anyone know how wxGtk must be built in order to make sure that library exists? Is there some special configure flag, maybe? Take care, Peter ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Problems with libgmp on Amazon Linux
Am 07.01.2012 20:01, schrieb Nabil Alsharif: Ahh your main issue is that --prefix should be /usr not /usr/lib It would be best to reinstall libgmp with the correct prefix. Oh, thanks, I will try this. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
On 01/07/2012 04:21 PM, Aristid Breitkreuz wrote: Yeah and that's annoying IMHO. :) It's not really important though. What would you prefer ? At the moment, i'm inclined to someday move cryptohash apis to be similar to crypto-api. i.e. from a result type being a bytestring to an opaque type with serialize/show instance. -- Vincent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
2012/1/8 Vincent Hanquez t...@snarc.org: What would you prefer ? At the moment, i'm inclined to someday move cryptohash apis to be similar to crypto-api. i.e. from a result type being a bytestring to an opaque type with serialize/show instance. Why? I don't actually need the hash object for anything, usually. All I need is the ByteString, and then I need to learn how to use the cereal package to get it... So talking purely about convenience, I would prefer hash :: HashType - InputData - ByteString. That would probably not be extensible enough, so I'm not sure how to optimally do it. Aristid ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
On Sat, Jan 7, 2012 at 10:12 PM, Aristid Breitkreuz arist...@googlemail.com wrote: 2012/1/8 Vincent Hanquez t...@snarc.org: What would you prefer ? At the moment, i'm inclined to someday move cryptohash apis to be similar to crypto-api. i.e. from a result type being a bytestring to an opaque type with serialize/show instance. Why? I don't actually need the hash object for anything, usually. All I need is the ByteString, and then I need to learn how to use the cereal package to get it... So talking purely about convenience, I would prefer hash :: HashType - InputData - ByteString. That would probably not be extensible enough, so I'm not sure how to optimally do it. You could use 'hash :: Hash h = Proxy h - ByteString - ByteString'. where 'data Proxy a = Proxy'. Defined for you in the 'tagged' package. Antoine ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] too many open files using snap
Hi haskellers, I wrote a web server using snap framework. Basically, it's an API server using hdbc-mysql to connect to a mysql server. But it also serves some static web pages and images, using the serveDirectory function in snap-core. The program eats more and more memory slowly, but I couldn't spot the leak. It uses less than 10MB memory when starting. Sometimes, it can grow to 60-70MB and stop working correctly. Then I find that the log file is full of too many open files error. As I don't have any code open any files, I suspect it's the serveDirectory leaking. But I don't have any expertise on the core of snap, so, could some one give me some advice on this problem? Thanks. Eric ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
Why? I don't actually need the hash object for anything, usually. All I need is the ByteString, and then I need to learn how to use the cereal package to get it... What would you think if Crypto.Classes exported Data.Serialize.encode? Or how about if it exported Antoine's hash suggestion (under a different name)? Cheers, Thomas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANN: leapseconds-announced-2012
Dear all, The International Earth Rotation and Reference Systems Service (IERS) recently announced[1] that a positive leap second will be introduced at the end of June 2012. Consequently I have updated the leapseconds-announced library[2]. The original announcement of leapseconds-announced can be found below. Best regards, Bjorn Buckwalter [1] http://hpiers.obspm.fr/eoppc/bul/bulc/bulletinc.dat [2] http://hackage.haskell.org/package/leapseconds-announced -- Forwarded message -- From: Bjorn Buckwalter bjorn.buckwal...@gmail.com Date: Sat, Jan 17, 2009 at 13:23 Subject: ANN: leapseconds-announced-2009 To: hask...@haskell.org, haskell-cafe@haskell.org Dear all, I'm pleased to announce the upload of the leapseconds-announced package[1] to Hackage. leapseconds-announced contains a single module and a single function implementing the Data.Time.Clock.TAI.LeapSecondTable interface (type). The documentation[2] for Data.Time.Clock.TAI.LeapSecondTable says No table is provided, as any program compiled with it would become out of date in six months and with that I have no objections. However, I frequently find myself needing a LeapSecondTable for a quick-and-dirty one-off analysis or simulation of the present or past. In these cases I've lazily used (const 33) (or more recently: (const 34)) as my LeapSecondTable. leapseconds-announced is a pragmatic, if imperfect, improvement over my past practices. It provides a LeapSecondTable with all leap seconds announced to date (hence the name). Once the IERS announces[3] another leap second the package will need an update and all code using it a recompile. While this precludes its use in long-running production applications it is eminently adequate for my one-off uses or for applications that can afford to recompile infrequently. While, in the words of the Data.Time.Clock.TAI documentation, most people won't need this module I hope it can be of utility to someone. Thanks, Bjorn Buckwalter [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/leapseconds-announced [2] http://www.haskell.org/ghc/docs/latest/html/libraries/time/Data-Time-Clock-TAI.html [3] http://hpiers.obspm.fr/eoppc/bul/bulc/bulletinc.dat ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] All the random packages
Hi, I came across the beautiful algorithm for O(1) generation of rational discrete distributions http://www.keithschwarz.com/darts-dice-coins/ and though it'd be fun to implement it in Haskell and put on Hackage, but then it turned out that there's about a dozen random number packages, so I'm not sure to which one to contribute, or better to roll out my own package and encourage authors of the other packages to reference it and fit under their interface. What do you think? Which random number packages are considered to be most mature or perhaps promising - i.e., contributing to which ones will bring the community the most benefit? Is there any effort on unifying them going on? -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [web-devel] [ANNOUNCE] First release of crypto-conduit
On Sun, Jan 8, 2012 at 12:39 AM, Thomas DuBuisson thomas.dubuis...@gmail.com wrote: Why? I don't actually need the hash object for anything, usually. All I need is the ByteString, and then I need to learn how to use the cereal package to get it... What would you think if Crypto.Classes exported Data.Serialize.encode? This suggestion is probably a win even if you do nothing else. Or how about if it exported Antoine's hash suggestion (under a different name)? Cheers, Thomas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] OpenAL bindings?
On Sat, Jan 7, 2012 at 1:26 PM, Yitzchak Gale g...@sefer.org wrote: Jason Dagit wrote: Looks like the repo [1] for the OpenAL bindings that Sven Panne created [2] is no longer available. I assume this is a result of The Great Server Outage of 2011 [3]. [1] http://darcs.haskell.org/packages/OpenAL/ [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/OpenAL [3] http://www.haskell.org/pipermail/haskell-cafe/2011-February/088829.html No, darcs.haskell.org is not the community server. It is a separate server that seems to be located at Galois. Ah yes, I always forget which servers are where. I see that the following darcs repositories are available on the community server: http://code.haskell.org/OpenAL/OpenAL http://code.haskell.org/OpenAL/ALUT Hope this helps, Thank you so much! Someone (I guess that's me) should update the wiki. Jason ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
