Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: igloo Type: feature request | Status: new Priority: high | Milestone: 7.8.1 Component: Compiler | Version: 7.0.3 Keywords:| Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown |Testcase: Blockedby:|Blocking: Related: 5877 | -+-- Changes (by parcs): * cc: patrick@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: igloo Type: feature request | Status: new Priority: high | Milestone: 7.8.1 Component: Compiler | Version: 7.0.3 Keywords:| Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown |Testcase: Blockedby:|Blocking: Related: 5877 | -+-- Changes (by WrenThornton): * cc: wren@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: igloo Type: feature request | Status: new Priority: high | Milestone: 7.8.1 Component: Compiler | Version: 7.0.3 Keywords:| Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown |Testcase: Blockedby:|Blocking: Related: 5877 | -+-- Changes (by igloo): * milestone: 7.6.1 = 7.8.1 Comment: Punting -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: igloo Type: feature request | Status: new Priority: high | Milestone: 7.6.1 Component: Compiler | Version: 7.0.3 Keywords:| Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown |Testcase: Blockedby:|Blocking: Related: 5877 | -+-- Changes (by Khudyakov): * cc: alexey.skladnoy@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: igloo Type: feature request | Status: new Priority: high | Milestone: 7.6.1 Component: Compiler | Version: 7.0.3 Keywords:| Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty: Unknown |Testcase: Blockedby:|Blocking: Related: 5877 | -+-- Changes (by simonpj): * owner: = igloo * difficulty: = Unknown Comment: Assigning to Ian to investigate/propose -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: Type: feature request | Status: new Priority: high | Milestone: 7.6.1 Component: Compiler | Version: 7.0.3 Keywords:| Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty:|Testcase: Blockedby:|Blocking: Related: 5877 | -+-- Changes (by reinerp): * related: = 5877 Comment: See also #5877 -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: Type: feature request | Status: new Priority: high | Milestone: 7.6.1 Component: Compiler | Version: 7.0.3 Keywords:| Os: Unknown/Multiple Architecture: Unknown/Multiple | Failure: None/Unknown Difficulty:|Testcase: Blockedby:|Blocking: Related:| -+-- Changes (by reinerp): * cc: reiner.pope@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals
-+--
Reporter: tibbe | Owner:
Type: feature request | Status: new
Priority: high | Milestone: 7.6.1
Component: Compiler | Version: 7.0.3
Keywords:| Os: Unknown/Multiple
Architecture: Unknown/Multiple | Failure: None/Unknown
Difficulty:|Testcase:
Blockedby:|Blocking:
Related:|
-+--
Comment(by PHO):
I want both solutions too, especially the solution 1.
What concerns me is that there seems no means of creating primitive byte-
array literals with TH. That is, the
[http://hackage.haskell.org/packages/archive/template-
haskell/2.6.0.0/doc/html/Language-Haskell-TH-Syntax.html#t:Lit Lit] type
currently only has a constructor {{{StringPrimL String}}} which represents
an {{{Addr#}}} literal ''encoded in UTF-8'', thus {{{unsafePackAddressLen
3 \NUL\NUL\NUL#}}} works but {{{unsafePackAddressLen 3 $(litE $
StringPrimL \NUL\NUL\NUL)}}} doesn't. So we probably need to make a
change to the type of {{{StringPrimL}}}:
{{{
data Lit = CharL Char
| StringL String
| ...
| StringPrimL [Word8] -- Raw, non-encoded ...# literal.
}}}
--
Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:12
GHC http://www.haskell.org/ghc/
The Glasgow Haskell Compiler
___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe |Owner: Type: feature request | Status: new Priority: high |Milestone: 7.6.1 Component: Compiler | Version: 7.0.3 Keywords:| Testcase: Blockedby:| Difficulty: Os: Unknown/Multiple | Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Changes (by igloo): * milestone: 7.4.1 = 7.6.1 Comment: punting -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe |Owner: Type: feature request | Status: new Priority: high |Milestone: 7.4.1 Component: Compiler | Version: 7.0.3 Keywords:| Testcase: Blockedby:| Difficulty: Os: Unknown/Multiple | Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Changes (by boris): * cc: lykahb@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe |Owner: Type: feature request | Status: new Priority: high |Milestone: 7.4.1 Component: Compiler | Version: 7.0.3 Keywords:| Testcase: Blockedby:| Difficulty: Os: Unknown/Multiple | Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Changes (by PHO): * cc: pho@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe |Owner: Type: feature request | Status: new Priority: high |Milestone: 7.4.1 Component: Compiler | Version: 7.0.3 Keywords:| Testcase: Blockedby:| Difficulty: Os: Unknown/Multiple | Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Comment(by tibbe): If solution two works I'm for it. It feels a bit complex but perhaps there's no better option. I'm against solution 1 as it would make most Haskell programs Template Haskell programs, as byte string literals are quite common, and that feels a bit too much to solve this problem. If we could have proper byte string literals (ala Python's b...) that would be even better but I guess that would be a too invasive change. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe |Owner: Type: feature request | Status: new Priority: high |Milestone: 7.4.1 Component: Compiler | Version: 7.0.3 Keywords:| Testcase: Blockedby:| Difficulty: Os: Unknown/Multiple | Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Comment(by simonpj): What's wrong with Template Haskell? The solution is so simple, it seems a shame not to use it, no? -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe |Owner: Type: feature request | Status: new Priority: high |Milestone: 7.4.1 Component: Compiler | Version: 7.0.3 Keywords:| Testcase: Blockedby:| Difficulty: Os: Unknown/Multiple | Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Changes (by simonmar): * priority: normal = high * milestone: = 7.4.1 Comment: We really should do this. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals
-+--
Reporter: tibbe |Owner:
Type: feature request | Status: new
Priority: high |Milestone: 7.4.1
Component: Compiler | Version: 7.0.3
Keywords:| Testcase:
Blockedby:| Difficulty:
Os: Unknown/Multiple | Blocking:
Architecture: Unknown/Multiple | Failure: None/Unknown
-+--
Comment(by simonmar):
Simon and I discussed this just now and found two further problems in
addition to the extra runtime length operation described above.
'''1. The RULES in `ByteString` for optimising literal strings currently
do not work'''
e.g. try
{{{
{-# LANGUAGE OverloadedStrings #-}
module Foo where
import Data.ByteString.Char8
foo = abc
}}}
the generated code does the translation to/from `String` at runtime.
Presumably this broke at some point, possibly due to things being inlined
at the wrong time and/or let-floating.
'''2. The RULES in `ByteString` never worked for non-ASCII strings'''
This is because non-ASCII strings are UTF-8-encoded by GHC and wrapped in
`unpackCStringUtf8`, not `unpackCString`, so the RULEs don't apply.
--
We have two solutions.
'''1. Use quasi-quoting to declare `ByteString` literals.'''
You would write something like
{{{
foo :: ByteString
foo = [bytes| ffa3c9db77 ]
}}}
and the code for `bytes` would translate this directly into a call to
`unsafePackAddress` passing a primitive literal string (`HsStringPrim`,
aka `...#`). Note that primitive literal strings are not UTF-8-encoded
by GHC, they are just 8-bit stripped (this happens in the lexer, of all
places).
This would be more convenient for encoding static data into the program
through `ByteString` literals.
'''2. Fix the RULES for `ByteString` literals.'''
Here's one way we could fix all the problems described above. It's a bit
tricky and requires changes in various places.
First, somewhere in `base` or `ghc-prim` we have
{{{
newtype String8 = String8 String
string8 :: String - String8
{-# INLINE [0] string8 #-}
string8 = String8
unpackCString :: Addr# - String
unpackCStringUtf8 :: Addr# - String
unpackCString8Len :: Addr# - Int# - String8
}}}
Next, GHC internally has RULEs that convert
{{{
string8 (unpackCStringUtf8 utf8stringlit#)
=== unpackCString8Len stringlit# len#
string8 (unpackCString asciistringlit#)
=== unpackCString8Len asciistringlit# len#
}}}
where `stringlit#` is made by decoding `utf8stringlit#` and stripping
out bits 8 and above from each character.
Then in `Data.ByteString.Char8` we have:
{{{
module Data.ByteString.Char8 where
packAddressLen :: Addr# - Int# - ByteString
pack8 :: String8 - ByteString
instance IsString ByteString where
fromString = pack . string8
{-# RULES pack8/unpackCString8
pack8 (unpackCString8 addr len) = packAddressLen addr len
#-}
}}}
--
Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:3
GHC http://www.haskell.org/ghc/
The Glasgow Haskell Compiler
___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe |Owner: Type: feature request | Status: new Priority: high |Milestone: 7.4.1 Component: Compiler | Version: 7.0.3 Keywords:| Testcase: Blockedby:| Difficulty: Os: Unknown/Multiple | Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Changes (by simonmar): * cc: dons, dcoutts (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
[GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: Type: feature request | Status: new Priority: normal| Component: Compiler Version: 7.0.3 |Keywords: Testcase:| Blockedby: Os: Unknown/Multiple |Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- GHC insert calls to `unpackCString#` to convert string literals to `String`s. Libraries like bytestring use rewrite rules to match on this call to optimize code like `pack (unpackCString# s)`. If GHC would instead use a version of unpackCString#, say unpackCStringLen#, that includes the (statically known) length, creating `ByteString`s from literals could be a constant time operation instead of a linear time operation. Another use case, which motivated this ticket, is appending string literals to builders (e.g. using `Data.Binary.Builder.fromByteString`). For small strings the most efficient way to append a string to the builder is to copy the statically allocated string directly into the builder's output buffer. If the string length was known statically, we could do this efficiently using `memcpy` or even using a small unrolled loop. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
Re: [GHC] #5218: Add unpackCStringLen# to create Strings from string literals
#5218: Add unpackCStringLen# to create Strings from string literals -+-- Reporter: tibbe | Owner: Type: feature request | Status: new Priority: normal| Component: Compiler Version: 7.0.3 |Keywords: Testcase:| Blockedby: Os: Unknown/Multiple |Blocking: Architecture: Unknown/Multiple | Failure: None/Unknown -+-- Changes (by tibbe): * cc: johan.tibell@… (added) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/5218#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler ___ Glasgow-haskell-bugs mailing list Glasgow-haskell-bugs@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs
