[issue9529] Make re match object iterable

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch -easy
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +easy -patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Mark Lawrence]

Change by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think MizardX means that match object should be iterable. This will allow 
sequence unpacking.

 import re
 m = re.match(r'(\w+):(\w+)', 'qwerty:asdfgh')
 k, v = m
 k
'qwerty'
 v
'asdfgh'

This idea looks reasonable to me. Here is simple preliminary patch which 
implements it.

--
keywords: +patch
nosy: +serhiy.storchaka
stage: needs patch - patch review
title: Converge re.findall and re.finditer - Make re match object iterable
type: behavior - enhancement
versions: +Python 3.5 -Python 3.2
Added file: http://bugs.python.org/file36192/re_matchobj_iterable.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Matthew Barnett]

Matthew Barnett added the comment:

Match objects have a .groups method:

 import re
 m = re.match(r'(\w+):(\w+)', 'qwerty:asdfgh')
 m.groups()
('qwerty', 'asdfgh')
 k, v = m.groups()
 k
'qwerty'
 v
'asdfgh'

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yes, but the purpose of this feature is to simplify the use of finditer() in 
the for loop.

 import re
 for k, v in re.finditer(r(\w+):?(\w+)?, ab:cd\nef\n):
... print(k, v)
... 
ab cd
ef None

Currently you should either unpack manually:

for m in re.finditer(...):
k, v = m.groups()
...

This way doesn't work well with comprehensions.

Or use the operator module:

import operator
for k, v in map(operator.methodcaller('groups'), re.finditer(...)):
...

This way is too verbose and unclear.

Sorry, previous version of the patch had reference leak.

--
Added file: http://bugs.python.org/file36193/re_matchobj_iterable.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___diff -r 4c1d543135ef Doc/library/re.rst
--- a/Doc/library/re.rstThu Jul 31 23:58:27 2014 -0500
+++ b/Doc/library/re.rstFri Aug 01 14:57:03 2014 +0300
@@ -873,6 +873,15 @@
if match:
process(match)
 
+Match objects are iterable and can be unpacked.  ``iter(match)`` is equivalent
+to ``iter(match.groups())``.  For example:
+
+[(k, v) for k, v in re.finditer(r(\w+):?(\w+)?, ab:cd, ef)]
+   [('ab', 'cd'), ('ef', None)]
+
+.. versionadded:: 3.5
+   Match objects are now iterable.
+
 Match objects support the following methods and attributes:
 
 
diff -r 4c1d543135ef Lib/test/test_re.py
--- a/Lib/test/test_re.py   Thu Jul 31 23:58:27 2014 -0500
+++ b/Lib/test/test_re.py   Fri Aug 01 14:57:03 2014 +0300
@@ -349,6 +349,18 @@
  (None, 'b', None))
 self.assertEqual(pat.match('ac').group(1, 'b2', 3), ('a', None, 'c'))
 
+def test_re_match_iter(self):
+pat = re.compile(r'(\w+):(\w+)?:(\w+)')
+m = pat.match('ab::cd=')
+it = iter(m)
+self.assertEqual(next(it), 'ab')
+self.assertEqual(next(it), None)
+self.assertEqual(next(it), 'cd')
+self.assertRaises(StopIteration, next, it)
+self.assertEqual(list(m), ['ab', None, 'cd'])
+x, y, z = m
+self.assertEqual((x, y, z), ('ab', None, 'cd'))
+
 def test_re_fullmatch(self):
 # Issue 16203: Proposal: add re.fullmatch() method.
 self.assertEqual(re.fullmatch(ra, a).span(), (0, 1))
@@ -896,6 +908,10 @@
 self.assertEqual([item.group(0) for item in iter],
  [::, ::])
 
+iter = re.finditer(r(\w+):?(\w+)?, ab:cd\nef\n)
+self.assertEqual([(k, v) for k, v in iter],
+ [('ab', 'cd'), ('ef', None)])
+
 def test_bug_926075(self):
 self.assertTrue(re.compile('bug_926075') is not
  re.compile(b'bug_926075'))
diff -r 4c1d543135ef Modules/_sre.c
--- a/Modules/_sre.cThu Jul 31 23:58:27 2014 -0500
+++ b/Modules/_sre.cFri Aug 01 14:57:03 2014 +0300
@@ -2080,6 +2080,31 @@
 }
 
 static PyObject*
+match_iter(MatchObject* self)
+{
+PyObject *result, *iter;
+Py_ssize_t index;
+
+result = PyTuple_New(self-groups-1);
+if (!result)
+return NULL;
+
+for (index = 1; index  self-groups; index++) {
+PyObject* item;
+item = match_getslice_by_index(self, index, Py_None);
+if (!item) {
+Py_DECREF(result);
+return NULL;
+}
+PyTuple_SET_ITEM(result, index-1, item);
+}
+
+iter = PyObject_GetIter(result);
+Py_DECREF(result);
+return iter;
+}
+
+static PyObject*
 match_groups(MatchObject* self, PyObject* args, PyObject* kw)
 {
 PyObject* result;
@@ -2478,7 +2503,7 @@
 0,  /* tp_clear */
 0,  /* tp_richcompare */
 0,  /* tp_weaklistoffset */
-0,  /* tp_iter */
+(getiterfunc)match_iter,/* tp_iter */
 0,  /* tp_iternext */
 match_methods,  /* tp_methods */
 match_members,  /* tp_members */
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Ezio Melotti]

Ezio Melotti added the comment:

See also #19536.

I still think that if we do something about these issues, we should try to be 
compatible with the regex module.

If we are going to add support for both iterability and __getitem__, they 
should be consistent, so that list(m) == [m[0], m[1], m[N]].
This means that m[0] should be equal to m.group(0), rather than m.group(1).

Currently the Match object of the regex module supports __getitem__ (with m[0] 
== m.group[0]) but is not iterable:
 m = regex.match('([^:]+): (.*)', 'foo: bar')
 m[0], m[1], m[2]
('foo: bar', 'foo', 'bar')
 len(m)
3
 list(m)
TypeError: '_regex.Match' object is not iterable

I can see different possible solutions:
1) do what regex does, have m[X] == m.group(X) and live with m[0] == m.group(0) 
(this means that unpacking will be _, key, value = m);
2) have m[0] == m.group(1), which makes unpacking easier, but is inconsistent 
with both m.group() and with what regex does; *
3) disregard regex compatibility and implement what we think is best;


* since regex already has a few incompatibilities with re, a global 
flag/function could be added to regex to make it behave like the re module 
(where possible).  If necessary, the re module could also include and ignore a 
similar flag/function.  This would make interoperability between the two easier.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think that if the regex module will be adopted in the stdlib, not all it's 
feature should be included. Regex is too complicated. In particular indexing 
looks confusing (due to ambiguity of starting index and redundant first item in 
unpacking). If we will not add support for indexing, there will no 
incompatibility.

There is yet one solution:

0) Reject both iterating and indexing.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Ezio Melotti]

Ezio Melotti added the comment:

That's indeed another valid solution, even though having indexing and 
iterability would be convenient (assuming we can figure out a reasonable way to 
implement them).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


Removed file: http://bugs.python.org/file36192/re_matchobj_iterable.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Mark Lawrence]

Mark Lawrence added the comment:

Why worry about the new regex module?  It doesn't appear to be any closer to 
getting into the stdlib than it was when #2636 was first opened on 15th April 
2008, so maybe Python 4.0?

--
nosy: +BreamoreBoy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9529] Make re match object iterable

[Ezio Melotti]

Ezio Melotti added the comment:

Even if it doesn't get included in the stdlib, some people might decide to 
switch from re to regex (or possibly vice versa) for their projects, so the 
closer they are the easier this will be.  Another reason is that afaik the 
authors of the regex module made a conscious effort to maintain compatibility 
with re, so returning the favor is the least we can do.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9529
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com