Fuzzyman wrote:
> Alex Martelli wrote:
> > Fuzzyman <[EMAIL PROTECTED]> wrote:
> >...
> > > > Open the file with 'rU' mode, and check the file object's newline
> > > > attribute.
> > >
>
> Just to confirm, for a UTF16 encoded file, the newlines attribute is
> ``None``.
>
Hmmm... having read t
Alex Martelli wrote:
> Fuzzyman <[EMAIL PROTECTED]> wrote:
>...
> > > Open the file with 'rU' mode, and check the file object's newline
> > > attribute.
> >
Just to confirm, for a UTF16 encoded file, the newlines attribute is
``None``.
All the best,
Fuzzyman
http://www.voidspace.org.uk/pyth
Alex Martelli wrote:
> Fuzzyman <[EMAIL PROTECTED]> wrote:
>...
> > I can't open with a codec unless an encoding is explicitly supplied. I
> > still want to detect UTF16 even if the encoding isn't specified.
> >
> > As I said, I ought to test this... Without metadata I wonder how Python
> > d
Fuzzyman <[EMAIL PROTECTED]> wrote:
...
> I can't open with a codec unless an encoding is explicitly supplied. I
> still want to detect UTF16 even if the encoding isn't specified.
>
> As I said, I ought to test this... Without metadata I wonder how Python
> determines it ?
It doesn't. Python
Arthur wrote:
> Arthur wrote:
> Is my premise that tokenizer needs universal newline support to be
> reliable correct?
>
> What else could put it out of sync with the complier?
Anybody out there?
Is my question, and the real world issue that provked it, unclear.
Is the answer too obvious?
Hav
Alex Martelli wrote:
> Fuzzyman <[EMAIL PROTECTED]> wrote:
>...
> > > Open the file with 'rU' mode, and check the file object's newline
> > > attribute.
> >
> > Do you know if this works for multi-byte encodings ? Do files have
>
> You mean when you open them with the codecs module?
>
No, if
Bengt Richter wrote:
> On 6 Feb 2006 06:35:14 -0800, "Fuzzyman" <[EMAIL PROTECTED]> wrote:
>
> >Hello all,
> >
> >I'm trying to detect line endings used in text files. I *might* be
> >decoding the files into unicode first (which may be encoded using
> >multi-byte encodings) - which is why I'm not
Fuzzyman <[EMAIL PROTECTED]> wrote:
...
> > Open the file with 'rU' mode, and check the file object's newline
> > attribute.
>
> Do you know if this works for multi-byte encodings ? Do files have
You mean when you open them with the codecs module?
> metadata associated with them showing the l
On 6 Feb 2006 06:35:14 -0800, "Fuzzyman" <[EMAIL PROTECTED]> wrote:
>Hello all,
>
>I'm trying to detect line endings used in text files. I *might* be
>decoding the files into unicode first (which may be encoded using
>multi-byte encodings) - which is why I'm not letting Python handle the
>line end
Arthur wrote:
> Alex Martelli wrote:
>
> I just got flummoxed by this issue, working with a (pre-alpha) package
> by very experienced Python programmers who sent file.readline to
> tokenizer.py without universal newline support. Went on a long (and
> educational) journey trying to figure out w
Alex Martelli wrote:
> Fuzzyman <[EMAIL PROTECTED]> wrote:
>
>
>>Hello all,
>>
>>I'm trying to detect line endings used in text files. I *might* be
>>decoding the files into unicode first (which may be encoded using
>
>
> Open the file with 'rU' mode, and check the file object's newline
> attri
Alex Martelli wrote:
> Fuzzyman <[EMAIL PROTECTED]> wrote:
>
> > Hello all,
> >
> > I'm trying to detect line endings used in text files. I *might* be
> > decoding the files into unicode first (which may be encoded using
>
> Open the file with 'rU' mode, and check the file object's newline
> attri
Alex Martelli wrote:
> Fuzzyman <[EMAIL PROTECTED]> wrote:
>
> > Hello all,
> >
> > I'm trying to detect line endings used in text files. I *might* be
> > decoding the files into unicode first (which may be encoded using
>
> Open the file with 'rU' mode, and check the file object's newline
> attri
Fuzzyman enlightened us with:
> This is what I came up with. [...] Comments/corrections welcomed.
You could use a little more comments in the code, but apart from that
it looks nice.
Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity,
Fuzzyman <[EMAIL PROTECTED]> wrote:
> Hello all,
>
> I'm trying to detect line endings used in text files. I *might* be
> decoding the files into unicode first (which may be encoded using
Open the file with 'rU' mode, and check the file object's newline
attribute.
> My worry is that if '\n' *do
Sybren Stuvel wrote:
> Fuzzyman enlightened us with:
> > My worry is that if '\n' *doesn't* signify a line break on the Mac,
> > then it may exist in the body of the text - and trigger ``ending =
> > '\n'`` prematurely ?
>
> I'd count the number of occurences of '\r\n', '\n' without a preceding
>
Sybren Stuvel wrote:
> Fuzzyman enlightened us with:
> > My worry is that if '\n' *doesn't* signify a line break on the Mac,
> > then it may exist in the body of the text - and trigger ``ending =
> > '\n'`` prematurely ?
>
> I'd count the number of occurences of '\r\n', '\n' without a preceding
>
Fuzzyman enlightened us with:
> My worry is that if '\n' *doesn't* signify a line break on the Mac,
> then it may exist in the body of the text - and trigger ``ending =
> '\n'`` prematurely ?
I'd count the number of occurences of '\r\n', '\n' without a preceding
'\r' and '\r' without following '\n
Hello all,
I'm trying to detect line endings used in text files. I *might* be
decoding the files into unicode first (which may be encoded using
multi-byte encodings) - which is why I'm not letting Python handle the
line endings.
Is the following safe and sane :
text = open('test.txt', 'rb').read
19 matches
Mail list logo
