Re: Creating slice notation from string
On Sep 2, 8:52 pm, Steven D'Aprano wrote: > On Wed, 02 Sep 2009 17:32:09 -0700, Bob van der Poel wrote: > > > Actually, nither this or Jan's latest is working properly. I don't know > > if it's the slice() function or what (I'm using python 2.5). But: > > > x = [1,2,3,4,5] > > slice_string="2" > > items = [int(n) if n else None for n in slice_string.split(":")] > > [slice(*items)] > > [1, 2] > > It's not clear what is input and what is output. I'm *guessing* that the > first four lines are input and the fifth is output. > > By the way, nice catch for the "else None". But why are you wrapping the > call to slice() in a list in the fourth line? > > I can't replicate your results. I get the expected results: > > >>> slice_string="2" > >>> items = [int(n) if n else None for n in slice_string.split(":")] > >>> [slice(*items)] > > [slice(None, 2, None)] > > exactly the same as: > > >>> slice(2) > > slice(None, 2, None) > > Testing this, I get the expected result: > > >>> x = [1,2,3,4,5] > >>> x[slice(*items)] > > [1, 2] > > which is exactly the same if you do this: > > >>> x[:2:] > > [1, 2] > > > not the expected: > > > [3] > > Why would you expect that? You can't get that result from a slice based > on 2 only. Watch: > > >>> x[2::] > [3, 4, 5] > >>> x[:2:] > [1, 2] > >>> x[::2] > > [1, 3, 5] > > There is no slice containing *only* 2 which will give you the result you > are asking for. You would need to do this: > > >>> x[2:3] > > [3] > > Perhaps what you are thinking of is *indexing*: > > >>> x[2] > > 3 > > but notice that the argument to list.__getitem__ is an int, not a slice, > and the result is the item itself, not a list. > > To get the behaviour you want, you need something more complicated: > > def strToSlice(s): > if ':' in s: > items = [int(n) if n else None for n in s.split(':')] > else: > if s: > n = int(s) > items = [n, n+1] > else: > items = [None, None, None] > return slice(*items) > > > Things like -1 don't work either. > > They do for me: > > >>> slice_string="2:-2" > >>> x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] > >>> items = [int(n) if n else None for n in slice_string.split(":")] > >>> x[ slice(*items) ] > [3, 4, 5, 6, 7, 8] > >>> x[2:-2] > > [3, 4, 5, 6, 7, 8] > > > I'm really not sure what's going on, but I suspect it's the way that > > slice() is getting filled when the slice string isn't a nice one with > > each ":" present? > > I think you're confused between __getitem__ with a slice argument and > __getitem__ with an int argument. > > -- > Steven Yes, you're right ... I'm confused on what results I expect! Sorry about that ... now that another day has passed it's all a bit more clear. I need to print out a number of these messages and write some test code. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Sep 2, 3:55 pm, bvdp wrote: > I'm trying to NOT create a parser to do this and I'm sure that > it's easy if I could only see the light! > > Is it possible to take an arbitrary string in the form "1:2", "1", > ":-1", etc. and feed it to slice() and then apply the result to an > existing list? > > For example, I have a normal python list. Let's say that x = [1,2,3,4] > and I have a string, call it "s', in the format "[2:3]". All I need to > do is to apply "s" to "x" just like python would do. > > I can, of course, convert "x" to a list with split(), convert the 2 > and 3 to ints, and then do something like: x[a:b] ... but I'd like > something more general. I think the answer is in slice() but I'm lost. > > Thanks. You might also consider looking at the operator module, which provides to potentially useful methods - itemgetter and getslice. Neither will directly use the string containing the bare slice notation, however they do provide another alternative to directly using eval. ~G -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Sep 2, 4:55 pm, bvdp wrote: > I'm trying to NOT create a parser to do this and I'm sure that > it's easy if I could only see the light! > Well, this is a nice puzzler, better than a sudoku. Maybe a quick parser with pyparsing will give you some guidance on how to do this without a parser library: from pyparsing import * # relevant punctuation, suppress after parsing LBR,RBR,COLON = map(Suppress,"[]:") # expression to parse numerics and convert to int's integer = Regex("-?\d+").setParseAction(lambda t: int(t[0])) # first try, almost good enough, but wrongly parses "[2]" -> [2::] sliceExpr = ( LBR + Optional(integer,default=None) + Optional(COLON + Optional(integer,default=None), default=None) + Optional(COLON + Optional(integer,default=None), default=None) + RBR ) # better, this version special-cases "[n]" -> [n:n+1] # otherwise, just create slice from parsed int's singleInteger = integer + ~FollowedBy(COLON) singleInteger.setParseAction(lambda t : [t[0],t[0]+1]) sliceExpr = ( LBR + (singleInteger | Optional(integer,default=None) + Optional(COLON + Optional(integer,default=None), default=None) + Optional(COLON + Optional(integer,default=None), default=None) ) + RBR ) # attach parse action to convert parsed int's to a slice sliceExpr.setParseAction(lambda t: slice(*(t.asList( tests = """\ [2] [2:3] [2:] [2::2] [-1:-1:-1] [:-1] [::-1] [:]""".splitlines() testlist = range(10) for t in tests: parsedSlice = sliceExpr.parseString(t)[0] print t, parsedSlice, testlist[parsedSlice] Prints: [2] slice(2, 3, None) [2] [2:3] slice(2, 3, None) [2] [2:] slice(2, None, None) [2, 3, 4, 5, 6, 7, 8, 9] [2::2] slice(2, None, 2) [2, 4, 6, 8] [-1:-1:-1] slice(-1, -1, -1) [] [:-1] slice(None, -1, None) [0, 1, 2, 3, 4, 5, 6, 7, 8] [::-1] slice(None, None, -1) [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] [:] slice(None, None, None) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] Yes, it is necessary to handle the special case of a "slice" that is really just a single index. If your list of parsed integers has only a single value n, then the slice constructor creates a slice (None,n,None). What you really want, if you want everything to create a slice, is to get slice(n,n+1,None). That is what the singleInteger special case does in the pyparsing parser. -- Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Wed, 02 Sep 2009 17:32:09 -0700, Bob van der Poel wrote: > Actually, nither this or Jan's latest is working properly. I don't know > if it's the slice() function or what (I'm using python 2.5). But: > > x = [1,2,3,4,5] > slice_string="2" > items = [int(n) if n else None for n in slice_string.split(":")] > [slice(*items)] > [1, 2] It's not clear what is input and what is output. I'm *guessing* that the first four lines are input and the fifth is output. By the way, nice catch for the "else None". But why are you wrapping the call to slice() in a list in the fourth line? I can't replicate your results. I get the expected results: >>> slice_string="2" >>> items = [int(n) if n else None for n in slice_string.split(":")] >>> [slice(*items)] [slice(None, 2, None)] exactly the same as: >>> slice(2) slice(None, 2, None) Testing this, I get the expected result: >>> x = [1,2,3,4,5] >>> x[slice(*items)] [1, 2] which is exactly the same if you do this: >>> x[:2:] [1, 2] > not the expected: > > [3] Why would you expect that? You can't get that result from a slice based on 2 only. Watch: >>> x[2::] [3, 4, 5] >>> x[:2:] [1, 2] >>> x[::2] [1, 3, 5] There is no slice containing *only* 2 which will give you the result you are asking for. You would need to do this: >>> x[2:3] [3] Perhaps what you are thinking of is *indexing*: >>> x[2] 3 but notice that the argument to list.__getitem__ is an int, not a slice, and the result is the item itself, not a list. To get the behaviour you want, you need something more complicated: def strToSlice(s): if ':' in s: items = [int(n) if n else None for n in s.split(':')] else: if s: n = int(s) items = [n, n+1] else: items = [None, None, None] return slice(*items) > Things like -1 don't work either. They do for me: >>> slice_string="2:-2" >>> x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] >>> items = [int(n) if n else None for n in slice_string.split(":")] >>> x[ slice(*items) ] [3, 4, 5, 6, 7, 8] >>> x[2:-2] [3, 4, 5, 6, 7, 8] > I'm really not sure what's going on, but I suspect it's the way that > slice() is getting filled when the slice string isn't a nice one with > each ":" present? I think you're confused between __getitem__ with a slice argument and __getitem__ with an int argument. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Wed, 02 Sep 2009 17:36:36 -0700, Bob van der Poel wrote: On Sep 2, 4:27 pm, Ethan Furman wrote: Bob van der Poel wrote: >>For a one-liner: >> x[slice(*map(int, x[1:-1].split(':')))] > Thanks. > Almost works :) > For s="[2]" and s="[1:2]" it's fine. But, if I have > s = "[:2]" then I get: x[slice(*[int(i) for i in s.strip("[]").split(":")])] > Traceback (most recent call last): > File "", line 1, in > ValueError: invalid literal for int() with base 10: '' > Similar problem with [2:]. > Ideas? try: start, end = s[1:-1].split(':') except ValueError: start = int(s[1:-1] # only one value specified end = start+1 start = int(start) if start else 0 end = int(end) if end else len(x) x[start:end] ~Ethan~ Yes ... I see. I'm thinking that eval() is looking very nice. If we do it the above way we also have to check for empties using things like [1::2] and I'm really getting confused about the possibilities :) How about: [untested] s = s[1:-1] # strip the brackets if s.count(':') == 0: return x[int(s)] while s.count(':') < 2: s += ':' start, stop, step = s.split(':') start = int(start) if start else 0 end = int(stop) if stop else len(x) step = int(step) if step else 1 return x[start:stop:step) ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Sep 2, 4:27 pm, Ethan Furman wrote: > Bob van der Poel wrote: > > > > >>For a one-liner: > > >> x[slice(*map(int, x[1:-1].split(':')))] > > > Thanks. > > > Almost works :) > > > For s="[2]" and s="[1:2]" it's fine. But, if I have > > > s = "[:2]" then I get: > > x[slice(*[int(i) for i in s.strip("[]").split(":")])] > > > Traceback (most recent call last): > > File "", line 1, in > > ValueError: invalid literal for int() with base 10: '' > > > Similar problem with [2:]. > > > Ideas? > > try: > start, end = s[1:-1].split(':') > except ValueError: > start = int(s[1:-1] # only one value specified > end = start+1 > start = int(start) if start else 0 > end = int(end) if end else len(x) > x[start:end] > > ~Ethan~ Yes ... I see. I'm thinking that eval() is looking very nice. If we do it the above way we also have to check for empties using things like [1::2] and I'm really getting confused about the possibilities :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Sep 2, 5:16 pm, Steven D'Aprano wrote: > On Wed, 02 Sep 2009 16:41:34 -0700, Bob van der Poel wrote: > > > But, translating 1, 2 or 3 ints into a valid splice isn't quit that > > easy? I could figure each value, and convert them to either int or None > > (key is the None! From my previous try '' doesn't work!) > > > But, I still need three possible lines: > > > if len(i) == 1: > > x=x[i(0)] > > else if len(i) == 2: > > x=x[i(0):i(1)] > > > > items = [int(n) for n in slice_string.split(":")] > slice(*items) > > -- > Steven Actually, nither this or Jan's latest is working properly. I don't know if it's the slice() function or what (I'm using python 2.5). But: x = [1,2,3,4,5] slice_string="2" items = [int(n) if n else None for n in slice_string.split(":")] [slice(*items)] [1, 2] not the expected: [3] Things like -1 don't work either. I'm really not sure what's going on, but I suspect it's the way that slice() is getting filled when the slice string isn't a nice one with each ":" present? -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
Bob van der Poel wrote: For a one-liner: x[slice(*map(int, x[1:-1].split(':')))] Thanks. Almost works :) For s="[2]" and s="[1:2]" it's fine. But, if I have s = "[:2]" then I get: x[slice(*[int(i) for i in s.strip("[]").split(":")])] Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '' Similar problem with [2:]. Ideas? try: start, end = s[1:-1].split(':') except ValueError: start = int(s[1:-1] # only one value specified end = start+1 start = int(start) if start else 0 end = int(end) if end else len(x) x[start:end] ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Wed, 02 Sep 2009 16:41:34 -0700, Bob van der Poel wrote: > But, translating 1, 2 or 3 ints into a valid splice isn't quit that > easy? I could figure each value, and convert them to either int or None > (key is the None! From my previous try '' doesn't work!) > > But, I still need three possible lines: > > if len(i) == 1: > x=x[i(0)] > else if len(i) == 2: > x=x[i(0):i(1)] > items = [int(n) for n in slice_string.split(":")] slice(*items) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Sep 2, 4:43 pm, "Jan Kaliszewski" wrote: > 03-09-2009 o 00:55:10 Bob van der Poel wrote: > > > > >> For a one-liner: > > >> x[slice(*map(int, x[1:-1].split(':')))] > > > Thanks. > > > Almost works :) > > > For s="[2]" and s="[1:2]" it's fine. But, if I have > > > s = "[:2]" then I get: > > x[slice(*[int(i) for i in s.strip("[]").split(":")])] > > Traceback (most recent call last): > > File "", line 1, in > > ValueError: invalid literal for int() with base 10: '' > > > Similar problem with [2:]. > > > Ideas? > > x = [1,4,3,5,4,6,5,7] > s = '[3:6]' > > x[slice(*((int(i) if i else None) > for i in s.strip("[]").split(":")))] > Thanks. I think this will work fine. If I paste the above in a python shell it's perfect. And if I paste it into my code it errors out ... so I have to look a bit more. Always the problem with one-liners if figuring out where the error is. I think it's just a naming thing ... I'm sure I'll get it soon. I'll shout if I find more problems. Best, -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Sep 2, 4:16 pm, "Rhodri James" wrote: > On Wed, 02 Sep 2009 23:57:48 +0100, Bob van der Poel > wrote: > > > > >> Of course, you could also do something like this: > > >> eval('x' + s) > >> or > >> eval(str(x) + s) > > > Yes, I have user inputed 's'. So, if I can't get the generalized list > > version from Robert working I'll have to use this. Speed is not a big > > deal in this. As to malicious input, I could pretty easily check to > > see that all the values are integers. > > If you've done that check, you've parsed the input so you might as well > use the values you've derived rather than waste time and add risk by > using eval(). Not sure exactly what you mean here? Checking the values is (fairly) trivial. Probably split on ":" and check the resulting list, etc. But, translating 1, 2 or 3 ints into a valid splice isn't quit that easy? I could figure each value, and convert them to either int or None (key is the None! From my previous try '' doesn't work!) But, I still need three possible lines: if len(i) == 1: x=x[i(0)] else if len(i) == 2: x=x[i(0):i(1)] But, I'm wondering if just doing a split() on the index list and seeing if they are all ints or None isn't safe? -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
03-09-2009 o 00:55:10 Bob van der Poel wrote: For a one-liner: x[slice(*map(int, x[1:-1].split(':')))] Thanks. Almost works :) For s="[2]" and s="[1:2]" it's fine. But, if I have s = "[:2]" then I get: x[slice(*[int(i) for i in s.strip("[]").split(":")])] Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '' Similar problem with [2:]. Ideas? x = [1,4,3,5,4,6,5,7] s = '[3:6]' x[slice(*((int(i) if i else None) for i in s.strip("[]").split(":")))] Cheers, *j -- Jan Kaliszewski (zuo) -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On 2009-09-02 17:55 PM, Bob van der Poel wrote: For a one-liner: x[slice(*map(int, x[1:-1].split(':')))] Thanks. Almost works :) For s="[2]" and s="[1:2]" it's fine. But, if I have s = "[:2]" then I get: x[slice(*[int(i) for i in s.strip("[]").split(":")])] Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '' Similar problem with [2:]. Ideas? Expanding out to a couple of lines now: slice_args = [] for i in s.strip("[]").split(':'): if i.strip() == '': slice_args.append(None) else: slice_args.append(int(i)) y = x[slice(*slice_args)] -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On Wed, 02 Sep 2009 23:57:48 +0100, Bob van der Poel wrote: Of course, you could also do something like this: eval('x' + s) or eval(str(x) + s) Yes, I have user inputed 's'. So, if I can't get the generalized list version from Robert working I'll have to use this. Speed is not a big deal in this. As to malicious input, I could pretty easily check to see that all the values are integers. If you've done that check, you've parsed the input so you might as well use the values you've derived rather than waste time and add risk by using eval(). -- Rhodri James *-* Wildebeest Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
> Of course, you could also do something like this: > > eval('x' + s) > or > eval(str(x) + s) > Yes, I have user inputed 's'. So, if I can't get the generalized list version from Robert working I'll have to use this. Speed is not a big deal in this. As to malicious input, I could pretty easily check to see that all the values are integers. I tried something like this earlier but could not get it to work. Didn't think of converting my x[] to a string. Makes sense. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
> For a one-liner: > > x[slice(*map(int, x[1:-1].split(':')))] Thanks. Almost works :) For s="[2]" and s="[1:2]" it's fine. But, if I have s = "[:2]" then I get: >>> x[slice(*[int(i) for i in s.strip("[]").split(":")])] Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '' Similar problem with [2:]. Ideas? -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
Erratum: eval(str(x) + s) -- but it's worse: less secure (e.g. if s could be user-typed) and most probably much more time-consuming (especially the latter). There should be *repr* instead of *str*. *j -- Jan Kaliszewski (zuo) -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
03-09-2009 o 00:11:17 MRAB wrote: bvdp wrote: I'm trying to NOT create a parser to do this and I'm sure that it's easy if I could only see the light! Is it possible to take an arbitrary string in the form "1:2", "1", ":-1", etc. and feed it to slice() and then apply the result to an existing list? For example, I have a normal python list. Let's say that x = [1,2,3,4] and I have a string, call it "s', in the format "[2:3]". All I need to do is to apply "s" to "x" just like python would do. I can, of course, convert "x" to a list with split(), convert the 2 and 3 to ints, and then do something like: x[a:b] ... but I'd like something more general. I think the answer is in slice() but I'm lost. >>> x = [1,2,3,4] >>> s = "[2:3]" >>> # Using map. >>> x[slice(*map(int, s.strip("[]").split(":")))] [3] >>> # Using a list comprehension. >>> x[slice(*[int(i) for i in s.strip("[]").split(":")])] [3] Of course, you could also do something like this: eval('x' + s) or eval(str(x) + s) -- but it's worse: less secure (e.g. if s could be user-typed) and most probably much more time-consuming (especially the latter). Cheers, *j -- Jan Kaliszewski (zuo) -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
On 2009-09-02 16:55 PM, bvdp wrote: I'm trying to NOT create a parser to do this and I'm sure that it's easy if I could only see the light! Is it possible to take an arbitrary string in the form "1:2", "1", ":-1", etc. and feed it to slice() and then apply the result to an existing list? For example, I have a normal python list. Let's say that x = [1,2,3,4] and I have a string, call it "s', in the format "[2:3]". All I need to do is to apply "s" to "x" just like python would do. I can, of course, convert "x" to a list with split(), convert the 2 and 3 to ints, and then do something like: x[a:b] ... but I'd like something more general. I think the answer is in slice() but I'm lost. For a one-liner: x[slice(*map(int, x[1:-1].split(':')))] -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating slice notation from string
bvdp wrote: I'm trying to NOT create a parser to do this and I'm sure that it's easy if I could only see the light! Is it possible to take an arbitrary string in the form "1:2", "1", ":-1", etc. and feed it to slice() and then apply the result to an existing list? For example, I have a normal python list. Let's say that x = [1,2,3,4] and I have a string, call it "s', in the format "[2:3]". All I need to do is to apply "s" to "x" just like python would do. I can, of course, convert "x" to a list with split(), convert the 2 and 3 to ints, and then do something like: x[a:b] ... but I'd like something more general. I think the answer is in slice() but I'm lost. >>> x = [1,2,3,4] >>> s = "[2:3]" >>> # Using map. >>> x[slice(*map(int, s.strip("[]").split(":")))] [3] >>> # Using a list comprehension. >>> x[slice(*[int(i) for i in s.strip("[]").split(":")])] [3] -- http://mail.python.org/mailman/listinfo/python-list