Re: [Haskell-cafe] FFI C Structs
On 7/18/08, Aleš Bizjak <[EMAIL PROTECTED]> wrote: > PS: An example of a haskell binding like this would be even more useful, if > anyone knows of such. I don't know that it's good for much else besides serving as an example of an FFI binding that uses structs (and possibly as an example of what not to do), but you could look at my half-finished GraphicsMagick binding: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hsmagick Cheers, Tim -- Tim Chevalier * http://cs.pdx.edu/~tjc * Often in error, never in doubt "Good memories make me feel bad / Bad memories make me feel small" --Dom Leone ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Trying to install cabal
On Sat, 2008-07-19 at 01:26 +0200, Ben Franksen wrote: > Henning Thielemann wrote: > > On Mon, 14 Jul 2008, Tillmann Rendel wrote: > > > >> Apropos cabal-install: can i make it build documentation during the > >> installation process and store them in some central location? > > > > I also wondered about that. Maybe a '--haddock' flag for 'cabal install' ? > > Everyone seems to ask the same (me too). See Ticket #206 > (http://hackage.haskell.org/trac/hackage/ticket/206). Additional comments > count as priority bumper! So! Who wants to implement it? :-) Volunteers please line up over here. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: Profiling nested case
Hi! I had to change code somewhat. Now I have a function like: worldScene point = case redSphere (0,50,0) 50 point of v@(Right _) -> v Left d1 -> case greenSphere (25,-250,0) 50 point of v@(Right _) -> v Left d2 -> Left $ d1 `min` d2 (Of course there could be more objects.) Any suggestion how could I make this less hard-coded? Something which would take a list of objects (object functions) and then return a Right if any object function return Right or a minimum value of all Lefts. But that it would have similar performance? If not on my GHC version (6.8.3) on something newer (which uses fusion or something). Is there some standard function for this or should I write my own recursive function to run over a list of object functions? But I am afraid that this will be hard to optimize for compiler. (It is important to notice that the order of executing object functions is not important so it could be a good candidate for parallelism.) Mitar ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] carry "state" around ....
On 2008 Jul 19, at 2:40, Galchin, Vasili wrote: My viewpoint is that the above "Internal members" must be "carried" around in a Haskell program. Am I correct?? If I am correct, then the Linux implementation of Posix AIO is not portable to say Solaris? In hindsight, if I am correct, it seems that You are correct --- but Solaris also has its own addenda, and its standard fields are not at the same offsets as in the Linux aiocb. The only safe way to do this is to use an opaque aiocb on the Haskell side and accessors in C via FFI. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] carry "state" around ....
On Sat, 2008-07-19 at 01:40 -0500, Galchin, Vasili wrote: > hello, > > > Following is more of a criticism of Linux implementation of the > Posix real-time extension of asynchronous I/O if not interesting > please skip. The central data structure for Posix AIO is an aiocb. In > any case, the Linux implementors added to the aiocb: [..] > My viewpoint is that the above "Internal members" must be "carried" > around in a Haskell program. Am I correct?? If I am correct, then the > Linux implementation of Posix AIO is not portable to say Solaris? In > hindsight, if I am correct, it seems that the Linux implementation of > AIO should use the ordered pair (pid, fd) to reference the "internal" > members and leave the "aiocb" "clean"? Although it is different between platforms it is still portable. When you allocate memory in C for the aiocb struct you would use sizeof(aiocb). That's portable even if the size is different on Linux vs Solaris. Then members are only accessed by name which is again portable. Your problem perhaps is that you're trying to convert an aiocb into a pure haskell version and convert it back and expect to retain all the information. I think that is a mistake. Don't pass the aiocb's by value, pass them by reference. Use a ForeignPtr and just access the members you need whenever you need them. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] carry "state" around ....
On Sat, 2008-07-19 at 10:45 -0400, Brandon S. Allbery KF8NH wrote: > > On 2008 Jul 19, at 2:40, Galchin, Vasili wrote: > > > My viewpoint is that the above "Internal members" must be "carried" > > around in a Haskell program. Am I correct?? If I am correct, then > > the Linux implementation of Posix AIO is not portable to say > > Solaris? In hindsight, if I am correct, it seems that > > > You are correct --- but Solaris also has its own addenda, and its > standard fields are not at the same offsets as in the Linux aiocb. > The only safe way to do this is to use an opaque aiocb on the Haskell > side and accessors in C via FFI. You can do field accessors using an FFI pre-processor like c2hs or hsc2hs which will calculate the correct field offsets for the current platform. No need for C wrappers. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ansi2html - one program, several issues
2008/7/19 Krzysztof Skrzętnicki <[EMAIL PROTECTED]>: > Hi all > > 1) Profiling shows that very simple functions are source of great memory and > time consumption. However, if I turn them off and simply print their input > arguments instead, the overall time and memory consumption doesn't change. > But now another function is acting badly. My guess: somehow the cost of > Parsec code is shifted into whatever function is using it's output. Let's > see: Are you using Parsec to parse the whole file ? Then your problem is there : Parsec needs to read and process the whole file before it can give us any output since it thinks it could have to give us an error instead and it can't be sure of that before he has read the whole thing... In your case, your problem is such that you would prefer to treat the file as a stream, isn't it ? There are some parser library that can give output lazily (look at polyparse flavour), another option would be to only use Parsec where you need it and just read and print the ordinary text for example. -- Jedaï ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: Help using Network.Curl
I tried to translate that using the Network.Curl docs and ended up with the
following code, but it's not sending the post data correctly (an ebay
api error re an "unsupported verb", which I am told means a malformed
request). Any ideas? The code from a perl tutorial (which works for
me, making me think that my problem is just about the right way to
post xml to a web service) follows the haskell version, which is meant
to be a straight translation of that.
Thanks,
Jim
--
[..skipped definitions of tokens etc..]
opts = [CurlEncoding "text/xml"
, CurlHttpHeaders ["X-EBAY-API-COMPATIBILITY-LEVEL="++compatLevel
, "X-EBAY-API-DEV-NAME="++devName
, "X-EBAY-API-APP-NAME="++appName
, "X-EBAY-API-CERT-NAME="++certName
, "X-EBAY-API-CALL-NAME=GeteBayOfficialTime"
, "X-EBAY-API-SITEID=0"]]
body = ""
++ ""
++ " "
++ ""++token++""
++ " "
++ ""
url = "https://api.sandbox.ebay.com/ws/api.dll";
postData = HttpPost {postName = "body"
, contentType = Just "text/xml"
, content = ContentString body
, extraHeaders = []
, showName = Nothing}
opts = []
--also tried using curlPost and setopts then perform...
main = withCurlDo (do h <- initialize
curlMultiPost url opts [postData]
)
---
Original perl:
use strict;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Headers;
# define the HTTP header
my $objHeader = HTTP::Headers->new;
$objHeader->push_header('X-EBAY-API-COMPATIBILITY-LEVEL' => '391');
$objHeader->push_header('X-EBAY-API-DEV-NAME' => 'yourdevname');
$objHeader->push_header('X-EBAY-API-APP-NAME' => 'yourappname');
$objHeader->push_header('X-EBAY-API-CERT-NAME' => 'yourcertname');
$objHeader->push_header('X-EBAY-API-CALL-NAME' => 'GeteBayOfficialTime');
$objHeader->push_header('X-EBAY-API-SITEID' => '0');
$objHeader->push_header('Content-Type' => 'text/xml');
# define the XML request
my $request =
"" .
"" .
" " .
"TOKENGoesHERE" .
" " .
"";
# make the call
my $objRequest = HTTP::Request->new(
"POST",
"https://api.sandbox.ebay.com/ws/api.dll";,
$objHeader,
$request
);
# deal with the response
my $objUserAgent = LWP::UserAgent->new;
my $objResponse = $objUserAgent->request($objRequest);
if (!$objResponse->is_error) {
print $objResponse->content;
}
else {
print $objResponse->error_as_HTML;
}
--
This is what I get when running my version:
$ ./curlTest
* About to connect() to api.sandbox.ebay.com port 443 (#0)
* Trying 66.135.197.133... * connected
* Connected to api.sandbox.ebay.com (66.135.197.133) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* SSL connection using AES256-SHA
* Server certificate:
*subject: /C=US/ST=California/L=San Jose/O=eBay Inc./OU=Site
Operations/CN=api.sandbox.ebay.com
*start date: 2008-01-25 00:00:00 GMT
*expire date: 2010-03-15 23:59:59 GMT
*common name: api.sandbox.ebay.com (matched)
*issuer: /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of
use at https://www.verisign.com/rpa (c)05/CN=VeriSign Class 3 Secure Server CA
* SSL certificate verify ok.
> POST /ws/api.dll HTTP/1.1
Host: api.sandbox.ebay.com
Accept: */*
Accept-Encoding: text/xml
Content-Length: 1254
Expect: 100-continue
Content-Type: multipart/form-data;
boundary=7c836e9408cd
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Date: Sat, 19 Jul 2008 20:04:12 GMT
< Server: Microsoft-IIS/5.0
< X-EBAY-API-POOL-NAME: ___cDRidW90YmtiZWx9b2l3aQ==
< X-EBAY-API-SERVER-NAME: ___dXUucnVlYnVvNzM3KTYzKzc3LTI0KTQxPD8zPTc=
< Content-Type: text/xml
< Content-Length: 330
< X-Cache: MISS from sjcsbagpigw02.sjc.ebay.com, MISS from
sjcsbagpigw02.sjc.ebay.com
< Connection: close
<
* Closing connection #0
2008-07-19
20:04:552RequestError1SeriousError00
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] carry "state" around ....
Brandon, You lost me can you explain what an opaque aiocb would look like? Vasili On Sat, Jul 19, 2008 at 9:45 AM, Brandon S. Allbery KF8NH < [EMAIL PROTECTED]> wrote: > > On 2008 Jul 19, at 2:40, Galchin, Vasili wrote: > > My viewpoint is that the above "Internal members" must be "carried" around > in a Haskell program. Am I correct?? If I am correct, then the Linux > implementation of Posix AIO is not portable to say Solaris? In hindsight, if > I am correct, it seems that > > > You are correct --- but Solaris also has its own addenda, and its standard > fields are not at the same offsets as in the Linux aiocb. The only safe way > to do this is to use an opaque aiocb on the Haskell side and accessors in C > via FFI. > > -- > brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] > system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] > electrical and computer engineering, carnegie mellon universityKF8NH > > > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ansi2html - one program, several issues
I forgot to mention that the memory consumption is several times higher than file size. On 8,3 Mb file: > 532 MB total memory in use (4 MB lost due to fragmentation). Having that 8 Mb in memory is not the problem. 532 Mb is another story. In general, the program consumes roughly 64 times more memory than file size and it scales linearly. Best regards Christopher Skrzętnicki On Sat, Jul 19, 2008 at 9:52 PM, Chaddaï Fouché <[EMAIL PROTECTED]> wrote: > 2008/7/19 Krzysztof Skrzętnicki <[EMAIL PROTECTED]>: > > Hi all > > > > 1) Profiling shows that very simple functions are source of great memory > and > > time consumption. However, if I turn them off and simply print their > input > > arguments instead, the overall time and memory consumption doesn't > change. > > But now another function is acting badly. My guess: somehow the cost of > > Parsec code is shifted into whatever function is using it's output. Let's > > see: > > Are you using Parsec to parse the whole file ? Then your problem is > there : Parsec needs to read and process the whole file before it can > give us any output since it thinks it could have to give us an error > instead and it can't be sure of that before he has read the whole > thing... > In your case, your problem is such that you would prefer to treat the > file as a stream, isn't it ? > There are some parser library that can give output lazily (look at > polyparse flavour), another option would be to only use Parsec where > you need it and just read and print the ordinary text for example. > > -- > Jedaï > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] ansi2html - one program, several issues
Hello Krzysztof, Sunday, July 20, 2008, 12:49:54 AM, you wrote: on the 32-bit computers 36x memreqs for storing large strings in memory is a rule, on 64-bit ones - 72x > I forgot to mention that the memory consumption is several times > higher than file size. On 8,3 Mb file: >> 532 MB total memory in use (4 MB lost due to fragmentation). > Having that 8 Mb in memory is not the problem. 532 Mb is another > story. In general, the program consumes roughly 64 times more memory > than file size and it scales linearly. > > Best regards > Christopher Skrzetnicki > On Sat, Jul 19, 2008 at 9:52 PM, Chaddai Fouche <[EMAIL PROTECTED]> wrote: > 2008/7/19 Krzysztof Skrzetnicki < [EMAIL PROTECTED]>: >> Hi all > >> >> 1) Profiling shows that very simple functions are source of great memory and >> time consumption. However, if I turn them off and simply print their input >> arguments instead, the overall time and memory consumption doesn't change. >> But now another function is acting badly. My guess: somehow the cost of >> Parsec code is shifted into whatever function is using it's output. Let's >> see: > > > Are you using Parsec to parse the whole file ? Then your problem is > there : Parsec needs to read and process the whole file before it can > give us any output since it thinks it could have to give us an error > instead and it can't be sure of that before he has read the whole > thing... > In your case, your problem is such that you would prefer to treat the > file as a stream, isn't it ? > There are some parser library that can give output lazily (look at > polyparse flavour), another option would be to only use Parsec where > you need it and just read and print the ordinary text for example. > -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] carry "state" around ....
On 2008 Jul 19, at 16:42, Galchin, Vasili wrote: You lost me can you explain what an opaque aiocb would look like? In Haskell, it's a ForeignPtr --- you can't see "inside" it except by FFI calls. Although Duncan is correct and you can use an FFI preprocessor to access the portable fields, and simply not provide access to the rest from Haskell. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ansi2html - one program, several issues
2008/7/19 Krzysztof Skrzętnicki <[EMAIL PROTECTED]>: > I forgot to mention that the memory consumption is several times higher than > file size. On 8,3 Mb file: >> 532 MB total memory in use (4 MB lost due to fragmentation). > > Having that 8 Mb in memory is not the problem. 532 Mb is another story. In > general, the program consumes roughly 64 times more memory than file size > and it scales linearly. You should be using ByteString, though this problem would be alleviated if you were consuming the file as a stream. -- Jedaï ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ansi2html - one program, several issues
On Sat, Jul 19, 2008 at 11:35 PM, Chaddaï Fouché <[EMAIL PROTECTED]> wrote: > 2008/7/19 Krzysztof Skrzętnicki <[EMAIL PROTECTED]>: > > I forgot to mention that the memory consumption is several times higher > than > > file size. On 8,3 Mb file: > >> 532 MB total memory in use (4 MB lost due to fragmentation). > > > > Having that 8 Mb in memory is not the problem. 532 Mb is another story. > In > > general, the program consumes roughly 64 times more memory than file size > > and it scales linearly. > > You should be using ByteString, though this problem would be > alleviated if you were consuming the file as a stream. > > Since ANSI color codes doesn't contain characters like newline or space, I have simply split input file into such lines. Now the whole program behaves much better: GC time is below 10% and memory consumption dropped to 74 Mb per thread. It's still a lot of memory though and it certainly holds much more than one line of text. Best regards Christopher Skrzętnicki ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] ansi2html - one program, several issues
Hello Krzysztof, Sunday, July 20, 2008, 1:55:45 AM, you wrote: >>> 532 MB total memory in use (4 MB lost due to fragmentation). i think that Parsec library should hold entire file in memory only when you use 'try' for whole file. otherwise it should omit data as proceeded -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: Re[2]: [Haskell-cafe] ansi2html - one program, several issues
On Sun, Jul 20, 2008 at 12:34 AM, Bulat Ziganshin <[EMAIL PROTECTED]> wrote: > Hello Krzysztof, > > Sunday, July 20, 2008, 1:55:45 AM, you wrote: > >>> 532 MB total memory in use (4 MB lost due to fragmentation). > > i think that Parsec library should hold entire file in memory only when > you use 'try' for whole file. otherwise it should omit data as > proceeded > > That's exactly what I thought. But even if I remove the only 'try' I use the memory consumption remains unchanged: C:\cygwin\home\Metharius\killer\KillerPy\ansi2html\ansi2html_old.exe duzy.log +RTS -sstderr File duzy.log processed. It took 5.046875s. File size was 4166578 characters. 3,950,649,704 bytes allocated in the heap 535,544,056 bytes copied during GC 117,603,408 bytes maximum residency (9 sample(s)) 1,647,828 bytes maximum slop 265 MB total memory in use (2 MB lost due to fragmentation) Generation 0: 7527 collections, 0 parallel, 0.86s, 0.86s elapsed Generation 1: 9 collections, 0 parallel, 0.80s, 0.81s elapsed INIT time0.02s ( 0.00s elapsed) MUT time3.20s ( 3.63s elapsed) GCtime1.66s ( 1.67s elapsed) EXIT time0.00s ( 0.00s elapsed) Total time4.88s ( 5.30s elapsed) %GC time 34.0% (31.6% elapsed) Alloc rate1,227,386,315 bytes per MUT second Productivity 65.7% of total user, 60.5% of total elapsed One more thing to note: with partial parsing there is no longer a difference between mapM_ and mapMPar. Best regards Christopher Skrzętnicki ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ghc 6.8.3 build error with __DISCARD__ linking problem, please help!
Hi, I am trying to build ghc 6.8.3 from source release, my currently installed version is ghc 6.4.2 (x86 linux binary release version), and I got: /usr/local/ghc/bin/ghc -o ghc-pkg.bin -H16m -O -cpp -Wall -fno-warn-name-shadowing -fno-warn-unused-matches -DUSING_COMPAT -i../../compat -ignore-package Cabal -Rghc-timing -package unix-L../../compat -lghccompat Main.o Version.o CRT_noglob.o Main.o: In function `__stginit_ZCMain': ghc13533.hc:(.text+0x4): undefined reference to `__DISCARD__' ghc13533.hc:(.text+0x9): undefined reference to `__DISCARD__' Main.o: In function `__stginit_Main': ghc13533.hc:(.text+0x18): undefined reference to `__DISCARD__' ghc13533.hc:(.text+0x1d): undefined reference to `__DISCARD__' Main.o: In function `__stginit_Main_': ghc13533.hc:(.text+0x2c): undefined reference to `__DISCARD__' Main.o:ghc13533.hc:(.text+0x3d): more undefined references to `__DISCARD__' follow collect2: ld returned 1 exit status <> Do you know how I can fix this problem? My ghc and gcc versions are: The Glorious Glasgow Haskell Compilation System, version 6.4.2 gcc (GCC) 4.1.1 20060724 (prerelease) (4.1.1-4pclos2007) Thanks! Di, Yu 7.19 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] carry "state" around ....
yes Duncan I am trying to pass-by-value. I am familiar with ForeignPtr; however, I don't comprehend what you and Brandon are suggesting to do. Could either of you provide a code illustration or point at existing code to illustrate your approach? Kind regards, Vasili On Sat, Jul 19, 2008 at 8:28 AM, Duncan Coutts <[EMAIL PROTECTED]> wrote: > > On Sat, 2008-07-19 at 01:40 -0500, Galchin, Vasili wrote: > > hello, > > > > > > Following is more of a criticism of Linux implementation of the > > Posix real-time extension of asynchronous I/O if not interesting > > please skip. The central data structure for Posix AIO is an aiocb. In > > any case, the Linux implementors added to the aiocb: > > [..] > > > My viewpoint is that the above "Internal members" must be "carried" > > around in a Haskell program. Am I correct?? If I am correct, then the > > Linux implementation of Posix AIO is not portable to say Solaris? In > > hindsight, if I am correct, it seems that the Linux implementation of > > AIO should use the ordered pair (pid, fd) to reference the "internal" > > members and leave the "aiocb" "clean"? > > Although it is different between platforms it is still portable. When > you allocate memory in C for the aiocb struct you would use > sizeof(aiocb). That's portable even if the size is different on Linux vs > Solaris. Then members are only accessed by name which is again portable. > > Your problem perhaps is that you're trying to convert an aiocb into a > pure haskell version and convert it back and expect to retain all the > information. I think that is a mistake. Don't pass the aiocb's by value, > pass them by reference. Use a ForeignPtr and just access the members you > need whenever you need them. > > Duncan > > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: Re[2]: [Haskell-cafe] ansi2html - one program, several issues
2008/7/20 Krzysztof Skrzętnicki <[EMAIL PROTECTED]>: > On Sun, Jul 20, 2008 at 12:34 AM, Bulat Ziganshin > <[EMAIL PROTECTED]> wrote: >> >> Hello Krzysztof, >> >> Sunday, July 20, 2008, 1:55:45 AM, you wrote: >> >>> 532 MB total memory in use (4 MB lost due to fragmentation). >> >> i think that Parsec library should hold entire file in memory only when >> you use 'try' for whole file. otherwise it should omit data as >> proceeded >> > > That's exactly what I thought. But even if I remove the only 'try' I use the > memory consumption remains unchanged: It's true, but in your case your output is almost the raw input data, which means that even without a noxious "try", you still have the whole file in memory. Well hopefully not with your latest code, which I would really like to see. -- Jedaï ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ansi2html - one program, several issues
On Sun, Jul 20, 2008 at 02:34:09AM +0400, Bulat Ziganshin wrote: > i think that Parsec library should hold entire file in memory only when > you use 'try' for whole file. otherwise it should omit data as > proceeded I do not believe that is the case, since the return type of runParser "Either ParseError a" means that before you can extract the result of the parse from the 'Right' branch, it must evaluate whether the result is 'Left' or 'Right' meaning it needs to parse the whole input in order to determine whether the parse was succesful. This was the reason I made frisby's main parsing routine just be (roughly) > runPeg :: P a -> String -> a so you have to do something explicit like > runPegMaybe :: P a -> String -> Maybe a > runPegMaybe p s = runPeg (fmap Just p // return Nothing) s to force strictness in the parsing. Though, perhaps parsec is doing something more clever. I do know it uses the one token lookahead trick to determine which branch to take on alternation, but I don't think that solves the issue with parsing the entire file.. John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] carry "state" around ....
In order to write portable code that accesses C structures, you need to write a c shim, or better, use the 'hsc2hs' tool to produce portable code. For an example, you can see my OpenSSL binding code in http://repetae.net/repos/ginsu/RSA.hsc in particular the 'createPkey' function. the #ptr construct gets a pointer to a member of a C structure, and #peek and #poke let you read and set members. You should use (#const sizeof(struct foo)) to determine how much memory you need to allocate for a structure. (unless the API you are binding specifies some other allocation method) John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ansi2html - one program, several issues
On Sunday 20 July 2008, John Meacham wrote: > I do not believe that is the case, since the return type of runParser > "Either ParseError a" means that before you can extract the result of > the parse from the 'Right' branch, it must evaluate whether the result > is 'Left' or 'Right' meaning it needs to parse the whole input in order > to determine whether the parse was succesful. > > This was the reason I made frisby's main parsing routine just be > (roughly) > > > runPeg :: P a -> String -> a > > so you have to do something explicit like > > > runPegMaybe :: P a -> String -> Maybe a > > runPegMaybe p s = runPeg (fmap Just p // return Nothing) s > > to force strictness in the parsing. > > Though, perhaps parsec is doing something more clever. I do know it uses > the one token lookahead trick to determine which branch to take on > alternation, but I don't think that solves the issue with parsing the > entire file.. It doesn't stop it from parsing the entire file strictly. However, what it does do is prevent the parser from backtracking out of arbitrary amounts of lookahead. So, unless you use try (which allows for lookahead), when any token is consumed by the parser, it can be garbage collected (assuming the parser is the only thing pointing to the token stream). So, it consumes its input strictly, but with limited overhead (ideally using try only for some small bounded lookahead where it's needed). By contrast, a naive parser combinator of the form: p = foo <|> bar -- or p = try foo <|> bar in parsec Might read the entire file into memory parsing foo, without any of it being garbage collected until completion, in case foo fails and a backtrack to bar is required. Of course, this all assumes that the input to the parser can both be lazily generated, and discarded in pieces (so, not the case if reading an entire file into a strict byte string, for instance). -- Dan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
