[issue46274] backslash creating statement out of nothing

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy:  -pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46274] backslash creating statement out of nothing

[Jeremy]

New submission from Jeremy :

A source of one or more backslash-escaped newlines, and one final newline, is 
not tokenized the same as a source where those lines are "manually joined".

The source
```
\
\
\

```
produces the tokens NEWLINE, ENDMARKER when piped to the tokenize module.

Whereas the source
```

```
produces the tokens NL, ENDMARKER.

What I expect is to receive only one NL token from both sources. As per the 
documentation "Two or more physical lines may be joined into logical lines 
using backslash characters" ... "A logical line that contains only spaces, 
tabs, formfeeds and possibly a comment, is ignored (i.e., no NEWLINE token is 
generated)"

And, because these logical lines are not being ignored, if there are 
spaces/tabs, INDENT and DEDENT tokens are also being unexpectedly produced.

The source
```
\

```
produces the tokens INDENT, NEWLINE, DEDENT, ENDMARKER.

Whereas the source (with spaces)
```

```
produces the tokens NL, ENDMARKER.

--
components: Parser
messages: 409811
nosy: lys.nikolaou, pablogsal, ucodery
priority: normal
severity: normal
status: open
title: backslash creating statement out of nothing
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com