Re: Is there a better way? [combining f-string, thousands separator, right align]
On 26/08/24 23:00, Dan Sommers via Python-list wrote:
On 2024-08-26 at 20:42:32 +1200,
dn via Python-list wrote:
and if we really want to go over-board:
RIGHT_JUSTIFIED = ">"
THOUSANDS_SEPARATOR = ","
s_format = F"{RIGHT_JUSTIFIED}{S_FIELD_WIDTH}{THOUSANDS_SEPARATOR}"
or (better) because right-justification is the default for numbers:
s_format = F"{S_FIELD_WIDTH}{THOUSANDS_SEPARATOR}"
To the extreme that if your user keeps fiddling with presentations (none
ever do, do they?), all settings to do with s_format could be added to a
config/environment file, and thus be even further separated from
program-logic!
And then you'll need a parser, many of whose Unique Challenges™ aren't
even apparent until you start parsing files from actual users, and
you'll still need some sort of fallback in the code anyway for the case
that s_format can't be parsed (for whatever reason).
Isn't a config file what just caused the global CrowdStrike outage? ;-)
That said, I understand that report generators are a thing, not to
mention RPG (https://en.wikipedia.org/wiki/IBM_RPG).
Okay, sorry; I'll just crawl back into the hole from whence I came.
Not at all. Please continue to question/ask/suggest!
This is a valid point. There are costs and benefits (trade-offs) to all
decisions!
That said, writing one's own parser would become a veritable can of
worms/rabbit hole. Here be dragons!
Similarly, explaining this takes longer than writing the example itself!
Older Windows users will know about .ini files, and Linux Admins are
familiar with .conf files. Many of us are already using JSON or YAML
formats. Any of these (and more) could be pressed into service, as
above. At the 'top end', there are also whole libraries devoted to
establishing application configuration or "environments": default
values, config files, command-line options, user-input...
Have switched to using Python-poetry, which replaces packaging methods
such as setuptools (as well as virtual-environment tools). It takes its
project configuration specifications from a pyproject.toml file. So, for
a few projects lately, I've been using .toml for application-config as
well. However, I have to say, this more from an attempt at consistency
than a decision of logic. (critique welcome)
That said, a setup.py configuration, took the form:
setup(
name='demo_project',
version='1.1.0',
packages=find_packages(),
install_requires=[
'requests',
'numpy',
...
],
entry_points={
...
Accordingly, it offers an example of the simplest format (for us), and
one which has a zero-learning pre-requisite. At execution-time, the
moment such a config is import-ed, a syntax-error will immediately bring
proceedings to a halt!
I have some stats-wonks as clients. They dabble in programming, but
(fortunately) realise their limitations. (usually!) The boss has had to
ban them from 'improving' my code ($paid to be an improvement on their
usual quality), but including a .py configuration/options file has
proven to be an honor-preserving compromise. Of course, they manage
their own runs, adjusting parameters as they go. So, any errors are
their own, and they can fix themselves (without anyone else knowing!).
Such would not work in many?most other environments - children: do not
try this at home!
An irritation for those of us who have to delve into projects after
they've been written, is a git-history full of the sorts of
user-tweaking changes vilified earlier. Putting user-config into a
separate file, even a separate sub-directory, makes it easy to spot
which updates to ignore, and thus, which to consider!
PS the reason why CrowdStrike was not the end of humanity as we know it,
(and only that of those who only know MSFT's eco-system) is because the
majority of the world's Internet servers run Linux - including Azure
(brings to mind the old saw: the package said "runs on Windows-95 or
better" so I installed it on Linux!)
Joking aside, we (virtuous ones) ALWAYS test BEFORE release. Correct?
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way? [combining f-string, thousands separator, right align]
On 2024-08-26 at 20:42:32 +1200,
dn via Python-list wrote:
> and if we really want to go over-board:
>
> >>> RIGHT_JUSTIFIED = ">"
> >>> THOUSANDS_SEPARATOR = ","
> >>> s_format = F"{RIGHT_JUSTIFIED}{S_FIELD_WIDTH}{THOUSANDS_SEPARATOR}"
>
> or (better) because right-justification is the default for numbers:
>
> >>> s_format = F"{S_FIELD_WIDTH}{THOUSANDS_SEPARATOR}"
>
>
> To the extreme that if your user keeps fiddling with presentations (none
> ever do, do they?), all settings to do with s_format could be added to a
> config/environment file, and thus be even further separated from
> program-logic!
And then you'll need a parser, many of whose Unique Challenges™ aren't
even apparent until you start parsing files from actual users, and
you'll still need some sort of fallback in the code anyway for the case
that s_format can't be parsed (for whatever reason).
Isn't a config file what just caused the global CrowdStrike outage? ;-)
That said, I understand that report generators are a thing, not to
mention RPG (https://en.wikipedia.org/wiki/IBM_RPG).
Okay, sorry; I'll just crawl back into the hole from whence I came.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way? [combining f-string, thousands separator, right align]
On 26/08/24 03:12, Gilmeh Serda via Python-list wrote:
Subject explains it, or ask.
This is a bloody mess:
s = "123456789" # arrives as str
f"{f'{int(s):,}': >20}"
' 123,456,789'
With recent improvements to the expressions within F-strings, we can
separate the string from the format required. (reminiscent of FORTRAN
which had both WRITE and FORMAT statements, or for that matter HTML
which states the 'what' and CSS the 'how')
Given that the int() instance-creation has a higher likelihood of
data-error, it is recommended that it be a separate operation for ease
of fault-finding - indeed some will want to wrap it with try...except.
>>> s = "123456789" # arrives as str
>>> s_int = int( s ) # makes the transformation obvious and distinct
>>> s_format = ">20," # define how the value should be presented
>>> F"{s_int:{s_format}}"
' 123,456,789'
Further, some of us don't like 'magic-constants', hence (previously):
>>> S_FIELD_WIDTH = 20
>>> s_format = F">{S_FIELD_WIDTH},"
and if we really want to go over-board:
>>> RIGHT_JUSTIFIED = ">"
>>> THOUSANDS_SEPARATOR = ","
>>> s_format = F"{RIGHT_JUSTIFIED}{S_FIELD_WIDTH}{THOUSANDS_SEPARATOR}"
or (better) because right-justification is the default for numbers:
>>> s_format = F"{S_FIELD_WIDTH}{THOUSANDS_SEPARATOR}"
To the extreme that if your user keeps fiddling with presentations (none
ever do, do they?), all settings to do with s_format could be added to a
config/environment file, and thus be even further separated from
program-logic!
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way? [combining f-string, thousands separator, right align]
On Sun, 25 Aug 2024 15:12:20 GMT Gilmeh Serda via Python-list wrote:
>Subject explains it, or ask.
>
>This is a bloody mess:
>
s = "123456789" # arrives as str
f"{f'{int(s):,}': >20}"
>' 123,456,789'
>
f"{s:>20}"
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way? [combining f-string, thousands separator, right align]
On 2024-08-25 16:12, Gilmeh Serda via Python-list wrote:
Subject explains it, or ask.
This is a bloody mess:
s = "123456789" # arrives as str
f"{f'{int(s):,}': >20}"
' 123,456,789'
You don't need to format twice; you can combine them:
>>> s = "123456789"
>>> f'{int(s): >20,}'
' 123,456,789'
or if you rely on default behaviour:
>>> f'{int(s):20,}'
' 123,456,789'
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way? [combining f-string, thousands separator, right align]
On Sun, 25 Aug 2024 15:12:20 GMT Gilmeh Serda via Python-list wrote:
>Subject explains it, or ask.
>
>This is a bloody mess:
>
s = "123456789" # arrives as str
f"{f'{int(s):,}': >20}"
>' 123,456,789'
>
Oops.. forgot comma
f"{int(s):>20,}"
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to create a list of None objects?
On Thu, 12 Aug 2021 09:57:33 +0100, Stephen Tucker declaimed the following: > ># Logic Effect > ># > ># [None * 8]TypeError: unsupported operand type(s) for *: ... > ># [(None) * 8] TypeError: unsupported operand type(s) for *: ... > ># [((None)) * 8]TypeError: unsupported operand type(s) for *: ... > ># [(None,) * 8] [(None, None, None, None, None, None, None, None)] > ># list ((None) * 8) TypeError: unsupported operand type(s) for *: ... > In all the above, you've put the *8 INSIDE the list structure. You "working" example is actually creating a TUPLE stored (as the only element) inside a LIST. >>> [None] * 8 [None, None, None, None, None, None, None, None] >>> Creates a LIST of 8 elements, each being None. -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to create a list of None objects?
Thanks for this feedback, Chris, Matthieu. Both are spot on - and thanks for the timing comparison, Matthieu. I suppose I didn't think to try the solution you suggest because I didn't think that I would end up with a single list, but 8 of them. OK, I'll stop wriggling. Stephen. On Thu, Aug 12, 2021 at 10:22 AM Matthieu Dartiailh wrote: > You can achieve the same result by writing: > [None] * 8 > > Comparing both cases in IPython I get: > > In [1]: %timeit list((None,)*8) > 110 ns ± 0.785 ns per loop (mean ± std. dev. of 7 runs, 1000 loops > each) > > In [2]: %timeit [None] * 8 > 88.2 ns ± 0.432 ns per loop (mean ± std. dev. of 7 runs, 1000 loops > each) > > So the list multiplication appears a bit faster. > > Best > > Matthieu > > Le 8/12/2021 à 10:57 AM, Stephen Tucker a écrit : > > Hi, > > > > I thought I'd share the following piece of code that I have recently > written > > (a) to check that what I have done is reasonable - even optimum, > > (b) to inform others who might be wanting to do similar things, and > > (c) to invite comment from the community. > > > > --- > > > > # > > > > # Yes: Create an empty list of Band Limits for this language > > > > # > > > > # Note. The rather complicated logic on the right-hand side of the > > > > # assignment below is used here because none of the following > > > > # alternatives had the desired effect: > > > > # > > > > # Logic Effect > > > > # > > > > # [None * 8]TypeError: unsupported operand type(s) for *: ... > > > > # [(None) * 8] TypeError: unsupported operand type(s) for *: ... > > > > # [((None)) * 8]TypeError: unsupported operand type(s) for *: ... > > > > # [(None,) * 8] [(None, None, None, None, None, None, None, None)] > > > > # list ((None) * 8) TypeError: unsupported operand type(s) for *: ... > > > > # > > > > diclll_BLim [thisISO_] = list ((None,) * 8) > > > > > > --- > > > > Thanks in anticipation. > > > > Stephen Tucker. > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to create a list of None objects?
You can achieve the same result by writing: [None] * 8 Comparing both cases in IPython I get: In [1]: %timeit list((None,)*8) 110 ns ± 0.785 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each) In [2]: %timeit [None] * 8 88.2 ns ± 0.432 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each) So the list multiplication appears a bit faster. Best Matthieu Le 8/12/2021 à 10:57 AM, Stephen Tucker a écrit : Hi, I thought I'd share the following piece of code that I have recently written (a) to check that what I have done is reasonable - even optimum, (b) to inform others who might be wanting to do similar things, and (c) to invite comment from the community. --- # # Yes: Create an empty list of Band Limits for this language # # Note. The rather complicated logic on the right-hand side of the # assignment below is used here because none of the following # alternatives had the desired effect: # # Logic Effect # # [None * 8]TypeError: unsupported operand type(s) for *: ... # [(None) * 8] TypeError: unsupported operand type(s) for *: ... # [((None)) * 8]TypeError: unsupported operand type(s) for *: ... # [(None,) * 8] [(None, None, None, None, None, None, None, None)] # list ((None) * 8) TypeError: unsupported operand type(s) for *: ... # diclll_BLim [thisISO_] = list ((None,) * 8) --- Thanks in anticipation. Stephen Tucker. -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to create a list of None objects?
On Thu, Aug 12, 2021 at 6:59 PM Stephen Tucker wrote: > > Hi, > > I thought I'd share the following piece of code that I have recently written > (a) to check that what I have done is reasonable - even optimum, > (b) to inform others who might be wanting to do similar things, and > (c) to invite comment from the community. > > --- > > # > > # Yes: Create an empty list of Band Limits for this language > > # > > # Note. The rather complicated logic on the right-hand side of the > > # assignment below is used here because none of the following > > # alternatives had the desired effect: > > # > > # Logic Effect > > # > > # [None * 8]TypeError: unsupported operand type(s) for *: ... > > # [(None) * 8] TypeError: unsupported operand type(s) for *: ... > > # [((None)) * 8]TypeError: unsupported operand type(s) for *: ... > > # [(None,) * 8] [(None, None, None, None, None, None, None, None)] > > # list ((None) * 8) TypeError: unsupported operand type(s) for *: ... > > # > > diclll_BLim [thisISO_] = list ((None,) * 8) > Why not just: [None] * 8 ? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Is there a better way to create a list of None objects?
Hi, I thought I'd share the following piece of code that I have recently written (a) to check that what I have done is reasonable - even optimum, (b) to inform others who might be wanting to do similar things, and (c) to invite comment from the community. --- # # Yes: Create an empty list of Band Limits for this language # # Note. The rather complicated logic on the right-hand side of the # assignment below is used here because none of the following # alternatives had the desired effect: # # Logic Effect # # [None * 8]TypeError: unsupported operand type(s) for *: ... # [(None) * 8] TypeError: unsupported operand type(s) for *: ... # [((None)) * 8]TypeError: unsupported operand type(s) for *: ... # [(None,) * 8] [(None, None, None, None, None, None, None, None)] # list ((None) * 8) TypeError: unsupported operand type(s) for *: ... # diclll_BLim [thisISO_] = list ((None,) * 8) --- Thanks in anticipation. Stephen Tucker. -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
On Apr 3, 12:26 pm, Alain Ketterlin wrote: > nn writes: > >> > for item in tag23gr: > >> > ... value, key = tuple(item) > >> > ... if(g23tag.get(key)): > >> > ... g23tag[key].append(value) > >> > ... else: > >> > ... g23tag[key] = [value] > > >> for item in tag23gr: > >> g23tag.setdefault(item[0],[]).append(item[1]) > > Or alternatively: > > > from collections import defaultdict > > g23tag = defaultdict(list) > > for item in tag23gr: > > g23tag[item[0]].append(item[1]) > > Very handy in that case, but in general I dislike the idea of silently > inserting a default value when the access is a read, e.g., in > x=g23tag[wrung]. Explicit is better than implicit, as they say. YMMV. > > -- Alain. Valid point. Preferred choice depends on the access patterns to the dict (e.g. one write and multiple reads, multiple writes and one loop over items, etc.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
nn writes: >> > for item in tag23gr: >> > ... value, key = tuple(item) >> > ... if(g23tag.get(key)): >> > ... g23tag[key].append(value) >> > ... else: >> > ... g23tag[key] = [value] >> >> for item in tag23gr: >> g23tag.setdefault(item[0],[]).append(item[1]) > Or alternatively: > > from collections import defaultdict > g23tag = defaultdict(list) > for item in tag23gr: > g23tag[item[0]].append(item[1]) Very handy in that case, but in general I dislike the idea of silently inserting a default value when the access is a read, e.g., in x=g23tag[wrung]. Explicit is better than implicit, as they say. YMMV. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
On Apr 3, 11:02 am, Alain Ketterlin wrote: > python writes: > > tag23gr is a list of lists each with two items. > > g23tag is an empty dictionary when I run the for loop below. > > When is is complete each key is a graphic name who's values are a list > > of tags. > > > for item in tag23gr: > > ... value, key = tuple(item) > > ... if(g23tag.get(key)): > > ... g23tag[key].append(value) > > ... else: > > ... g23tag[key] = [value] > > for item in tag23gr: > g23tag.setdefault(item[0],[]).append(item[1]) > > -- Alain. Or alternatively: from collections import defaultdict g23tag = defaultdict(list) for item in tag23gr: g23tag[item[0]].append(item[1]) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
python wrote: > I played around with a few things and this works but was wondering if > there was a better way to do this. > My first thought was list comprehension but could not get a figure out > the syntax. > > tag23gr is a list of lists each with two items. > g23tag is an empty dictionary when I run the for loop below. > When is is complete each key is a graphic name who's values are a list > of tags. > > for item in tag23gr: > ... value, key = tuple(item) > ... if(g23tag.get(key)): That should be if key in g23tag: Your version means trouble for keys that evaluate to False in a boolean context, e. g. 0, False, None, "", (),... > ... g23tag[key].append(value) > ... else: > ... g23tag[key] = [value] from collections import defaultdict g23tag = defaultdict(list) for value, key in tag23gr: g23tag[key].append(value) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
On Wed, Apr 4, 2012 at 12:36 AM, python wrote: > for item in tag23gr: > ... value, key = tuple(item) > ... if(g23tag.get(key)): > ... g23tag[key].append(value) > ... else: > ... g23tag[key] = [value] Simple enhancement: Use setdefault. Instead of the if, just use: g23tag.setdefault(key,[]).append(value) That'll cover both cases in one. You can leave off the explicit tuple construction; if item is a two-element list, you can unpack it directly. You can also embed that straight into your for loop: for value,key in tag23gr: Do both and you cut your loop down to two lines. Cool! :) Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
python writes: > tag23gr is a list of lists each with two items. > g23tag is an empty dictionary when I run the for loop below. > When is is complete each key is a graphic name who's values are a list > of tags. > > for item in tag23gr: > ... value, key = tuple(item) > ... if(g23tag.get(key)): > ... g23tag[key].append(value) > ... else: > ... g23tag[key] = [value] for item in tag23gr: g23tag.setdefault(item[0],[]).append(item[1]) -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Is there a better way to do this snippet?
I played around with a few things and this works but was wondering if there was a better way to do this. My first thought was list comprehension but could not get a figure out the syntax. tag23gr is a list of lists each with two items. g23tag is an empty dictionary when I run the for loop below. When is is complete each key is a graphic name who's values are a list of tags. for item in tag23gr: ... value, key = tuple(item) ... if(g23tag.get(key)): ... g23tag[key].append(value) ... else: ... g23tag[key] = [value] -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to solve this?
On Mon, 23 May 2011 11:55:08 -0700, kracekumar ramaraju wrote: > You can use sizeof function, Who are you talking to, and what question did they ask? Please always quote enough of the post that you are replying to to establish context. a=12234 b=23456.8 a.__sizeof__() > 12 b.__sizeof__() > 16 > So sizeof int is 12 bytes and float is 16 bytes You shouldn't be calling special methods directly (except under unusual circumstances). That's like calling s.__len__() instead of len(s). The public function for getting the size of an object is in the sys module: sys.getsizeof(a) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to solve this?
On 5/23/2011 2:55 PM, kracekumar ramaraju wrote: You can use sizeof function, Appears not to be in manuals, that I could find. As a special method, it is intended to be called through sys.getsizeof. a=12234 b=23456.8 a.__sizeof__() 12 b.__sizeof__() 16 So sizeof int is 12 bytes and float is 16 bytes Depends on system. On my winxp machine, ints are 14 bytes. >>> import sys >>> size = sys.getsizeof >>> size(1) 14 >>> size(1.0) 16 >>> size([]) 36 >>> size([1,2,3]) 48 -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to solve this?
On Mon, May 23, 2011 at 2:55 PM, kracekumar ramaraju wrote: > You can use sizeof function, a=12234 b=23456.8 a.__sizeof__() > 12 b.__sizeof__() > 16 > So sizeof int is 12 bytes and float is 16 bytes I'm not sure what you're trying to show here, but try the following in Python 3.2 >>> a = >>> for i in range(5): ... a*= 10 ... a.__sizeof__() ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to solve this?
You can use sizeof function, >>> a=12234 >>> b=23456.8 >>> a.__sizeof__() 12 >>> b.__sizeof__() 16 So sizeof int is 12 bytes and float is 16 bytes -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to solve this?
On Sat, May 21, 2011 at 11:28 PM, Ganapathy Subramanium
wrote:
> Hello,
>
> I'm a new bie to python programming and on the processing of learning python
> programming. I have coded my first program of fibonnaci generation and would
> like to know if there are better ways of achieving the same.
>
> I still feel quite a few things to be improved. Just wanted experts thoughts
> on this.
>
> try:
>
> length = input('Enter the length till which you want to generate the
> fibonnaci series: \n')
> print type(length)
>
> except:
> print 'Invalid input detected'
> exit(0)
Never use the input() function in Python 2.x; it does an eval(), which
is evil. Always use raw_input() and an explicit conversion instead;
e.g.
Q = 'Enter the length till which you want to generate the fibonnaci series: \n'
try:
length = int(raw_input(Q))
except ValueError:
print 'Invalid input detected'
exit(0)
Also, as a sidenote:
> if type(length) is int:
Is normally written:
if isinstance(length, int):
Cheers,
Chris
--
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Is there a better way to solve this?
Hello,
I'm a new bie to python programming and on the processing of learning python
programming. I have coded my first program of fibonnaci generation and would
like to know if there are better ways of achieving the same.
I still feel quite a few things to be improved. Just wanted experts thoughts
on this.
try:
length = input('Enter the length till which you want to generate the
fibonnaci series: \n')
print type(length)
except:
print 'Invalid input detected'
exit(0)
if type(length) is int:
result = []
a = 0
b = 1
result.append(a)
result.append(b)
if length > 2:
count = 2
while count < length:
c = a + b
a = b
b = c
result.append(c)
count += 1
else:
print result
print 'The lenght of result is : ' , len(result)
else:
print 'Fibonacci series could not be generated !!!'
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to set a system clock in Python (on a Linux system)
In message , J wrote: > Like I said, it works well, I just wonder if there is a cleaner way of > setting the local clock to a different time in python without having > to do all this. How about one line in Bash: date -s $(date --rfc-3339=date -d "+1 hour") -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to set a system clock in Python (on a Linux system)
man 2 clock_settime
call it with ctypes
--
Best Regards,
-- KDr2 http://kdr2.net
On Thu, May 6, 2010 at 10:47 AM, J wrote:
> Is there a better way to do this?
>
> def SkewTime():
>'''
>Optional function. We can skew time by 1 hour if we'd like to see real
> sync
>changes being enforced
>'''
>TIME_SKEW=1
>logging.info('Time Skewing has been selected. Setting clock ahead 1
> hour')
># Let's get our current time
>t = TimeCheck()
>logging.info('Current time is: %s' % time.asctime(t))
># Now create new time string in the form MMDDhhmm for the date
> program
>hr = t.tm_hour + TIME_SKEW
>date_string = time.strftime('%m%d%H%M%Y',(t.tm_year,
>t.tm_mon,
>t.tm_mday,
>hr,
>t.tm_min,
>t.tm_sec,
>t.tm_wday,
>t.tm_yday,
>t.tm_isdst))
>logging.debug('New date string is: %s' % date_string)
>logging.debug('Setting new system time/date')
>status = SilentCall('/bin/date %s' % date_string)
>logging.info('Pre-sync time is: %s' % time.asctime())
>
> TimeCheck() as referenced above is a simple function that just returns
> the time.time_struct object from time.localtime(). I pull time a few
> times and it was a little cleaner to put that into a function and just
> call the function whenever I needed to.
>
> SilentCall() is a modification of subprocess.call() (which in reality
> just calls Popen(*popenargs,**kwargs).wait()) but it defaults to
> redirecting stdin and stdout to /dev/null to suppress shell output
> from the command being called.
>
> Anyway, what I'm wondering, is, while this works, is there a better
> way to do it than using part of the originally returned time_struct
> and injecting my own new hour argument (hr).
>
> The goal of this function is to just set the system clock one hour
> ahead, so when I call the Linux command 'ntpdate' I can get a real
> time change when it syncs the local clock to an NTP server.
>
> This just looks... well, big to me. I tried passing only the things I
> really needed to time.strftime(), but apparently, that requires the
> full 9-tuple from time_struct, not just individual parts of it.
>
> Like I said, it works well, I just wonder if there is a cleaner way of
> setting the local clock to a different time in python without having
> to do all this. The reason most of that exists, is because the linux
> date command expects to see the new date/time like this:
> MMDDhhmm.ss.
>
> Or am I just looking at this too hard and really did work it out nicely?
>
> Cheers
> Jeff
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to set a system clock in Python (on a Linux system)
On Wed, May 5, 2010 at 7:47 PM, J wrote:
> Is there a better way to do this?
Yes:
from datetime import datetime, timedelta
> def SkewTime():
> '''
> Optional function. We can skew time by 1 hour if we'd like to see real sync
> changes being enforced
> '''
> TIME_SKEW=1
> logging.info('Time Skewing has been selected. Setting clock ahead 1 hour')
> # Let's get our current time
skewed = datetime.now() + timedelta(hours=TIME_SKEW)
> # Now create new time string in the form MMDDhhmm for the date program
date_time_str = skewed.strftime('%m%d%H%M%Y')
logging.debug('New date string is: %s' % date_time_str)
> logging.debug('Setting new system time/date')
status = SilentCall('/bin/date %s' % date_time_str)
> logging.info('Pre-sync time is: %s' % time.asctime())
>
> Anyway, what I'm wondering, is, while this works, is there a better
> way to do it than using part of the originally returned time_struct
> and injecting my own new hour argument (hr).
Use the datetime module roughly as shown. (Disclaimer: Code is untested).
Also, I'm not sure if your original code worked properly after 11PM;
my code definitely should.
> This just looks... well, big to me. I tried passing only the things I
> really needed to time.strftime(), but apparently, that requires the
> full 9-tuple from time_struct, not just individual parts of it.
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Is there a better way to set a system clock in Python (on a Linux system)
Is there a better way to do this?
def SkewTime():
'''
Optional function. We can skew time by 1 hour if we'd like to see real sync
changes being enforced
'''
TIME_SKEW=1
logging.info('Time Skewing has been selected. Setting clock ahead 1 hour')
# Let's get our current time
t = TimeCheck()
logging.info('Current time is: %s' % time.asctime(t))
# Now create new time string in the form MMDDhhmm for the date program
hr = t.tm_hour + TIME_SKEW
date_string = time.strftime('%m%d%H%M%Y',(t.tm_year,
t.tm_mon,
t.tm_mday,
hr,
t.tm_min,
t.tm_sec,
t.tm_wday,
t.tm_yday,
t.tm_isdst))
logging.debug('New date string is: %s' % date_string)
logging.debug('Setting new system time/date')
status = SilentCall('/bin/date %s' % date_string)
logging.info('Pre-sync time is: %s' % time.asctime())
TimeCheck() as referenced above is a simple function that just returns
the time.time_struct object from time.localtime(). I pull time a few
times and it was a little cleaner to put that into a function and just
call the function whenever I needed to.
SilentCall() is a modification of subprocess.call() (which in reality
just calls Popen(*popenargs,**kwargs).wait()) but it defaults to
redirecting stdin and stdout to /dev/null to suppress shell output
from the command being called.
Anyway, what I'm wondering, is, while this works, is there a better
way to do it than using part of the originally returned time_struct
and injecting my own new hour argument (hr).
The goal of this function is to just set the system clock one hour
ahead, so when I call the Linux command 'ntpdate' I can get a real
time change when it syncs the local clock to an NTP server.
This just looks... well, big to me. I tried passing only the things I
really needed to time.strftime(), but apparently, that requires the
full 9-tuple from time_struct, not just individual parts of it.
Like I said, it works well, I just wonder if there is a cleaner way of
setting the local clock to a different time in python without having
to do all this. The reason most of that exists, is because the linux
date command expects to see the new date/time like this:
MMDDhhmm.ss.
Or am I just looking at this too hard and really did work it out nicely?
Cheers
Jeff
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this?
"Matt Mitchell" wrote in message news:mailman.65.1267464765.23598.python-l...@python.org... > My initial idea was to make a list of all the different > ways "project" has been capitalized in my repo and try each one. The > code looks like this: I would use pysvn.Client.list to get a list of files at whatever directory level you require. Then you can do a case insensitive compare or whatever else. -- http://mail.python.org/mailman/listinfo/python-list
Is there a better way to do this?
Hi,
I wrote a python script that uses pysvn to export projects from an svn
repo I have. The repo has hundreds of projects in it with a directory
structure that is pretty uniform however it's not exactly uniform
because of the capitalization. I.e.:
\root
\project English
\Stuff
\Stuff 2
\Project Spanish
\Stuff 3
\Stuff 4
My svn repo is case sensitive so if I try to get \root\project
Spanish\Stuff 3 I get an error. Fixing the capitalization is not an
option for me. My initial idea was to make a list of all the different
ways "project" has been capitalized in my repo and try each one. The
code looks like this:
import pysvn
def getstuff(stuffiwant, languageiwantitin):
projects = ("project %s/", "Project %s/", "pRojects %s/")
c = pysvn.Client()
for p in projects:
exportme = p % languageiwantitin
exportme = "http://localhost/"; + exportme + stuffiwant
try:
c.export(exportme, "C:\\temp\\")
break
except pysvn.ClientError:
print "Not the right capitalization."
# do the rest of the stuff I need to do.
This works, but to me it seems like there has to be a better way of
doing it. Any feedback or suggestions would be appreciated.
Thanks,
Matt
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
On Thursday, 8 October 2009 18:41:31 Dr. Phillip M. Feldman wrote: > I currently have a function that uses a list internally but then returns > the list items as separate return > values as follows: > > if len(result)==1: return result[0] > if len(result)==2: return result[0], result[1] > > (and so on). Is there a cleaner way to accomplish the same thing? Why do you not change the list into a tuple and return the tuple, and let automatic unpacking handle it? As I see it, the problem is not in the return, but in the call - how do you know now, which of the following to write: answer = thing(params) answer0,answer1 = thing(params) answer0,answer1,answer2 = thing(params) answer0,answer1,answer2,answer3 = thing(params) and so on... probably best to write: answers = thing(params) for answer in answers: do something with answer - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
On Thu, Oct 8, 2009 at 7:14 PM, Dr. Phillip M. Feldman wrote: > > I'm amazed that this works. I had not realized that > > x,y= [3,4] > > is equivalent to > > x= 3; y= 4 > > Python is rather clever. > > Thanks! > Python is very clever: >>> (a, b), c = (1, 2), 3 >>> a, b, c (1, 2, 3) :D -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Dr. Phillip M. Feldman writes: I currently have a function that uses a list internally but then returns the list items as separate return values as follows: if len(result)==1: return result[0] if len(result)==2: return result[0], result[1] (and so on). Is there a cleaner way to accomplish the same thing? The suggestions to return result or if needed tuple(result) are good, if a sequence is expected. But perhaps better if each element has separate meaning: return a defaultdict; document the keys. http://docs.python.org/library/collections.html#collections.defaultdict Cheers, Alan Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Dr. Phillip M. Feldman wrote: I'm amazed that this works. I had not realized that x,y= [3,4] is equivalent to x= 3; y= 4 Python is rather clever. Thanks! To elaborate on Paul's answer, returning the list will also unpack it if you have it set up that way. E.g. def func(alist): return alist some_list = [1, 2] this, that = func(alist) At least, in 2.5.4 this works. :-) In just about all Python versions for all sequence types, in fact. Mind you, if you don't have the correct number of return names to match the unpacking you'll get the normal errors from that. Yes. This is why people are suggesting that you be consistent about what you return. This is quite different from Matlab where the interpreter knows how many return values the caller is expecting in order to overload functions, but I think it makes for much more understandable code. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
This is an interesting alternative. If one wants to generate everything and return it at one shot, the list approach is better, but there are situations where generating things incrementally is preferrable, e.g., because the caller doesn't know a priori how many things he wants. I will try this out. Thanks! Jack Norton wrote: Dr. Phillip M. Feldman wrote: I currently have a function that uses a list internally but then returns the list items as separate return values as follows: if len(result)==1: return result[0] if len(result)==2: return result[0], result[1] (and so on). Is there a cleaner way to accomplish the same thing? How about using yield and then iterate over the answer: def some_fun(): \tfor result in some_loopable_stuff: \t\t yield result Then call it thusly: for i in some_fun() result = i -Jack (PS, sorry to the OP, you will get two of these -- I forgot to CC the list) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
I'm amazed that this works. I had not realized that x,y= [3,4] is equivalent to x= 3; y= 4 Python is rather clever. Thanks! To elaborate on Paul's answer, returning the list will also unpack it if you have it set up that way. E.g. def func(alist): return alist some_list = [1, 2] this, that = func(alist) At least, in 2.5.4 this works. :-) Mind you, if you don't have the correct number of return names to match the unpacking you'll get the normal errors from that. Hope this helps! ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list -- View this message in context: http://www.nabble.com/Is-there-a-better-way-to-code-variable-number-of-return-arguments--tp25803294p25813206.html Sent from the Python - python-list mailing list archive at Nabble.com. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Dr. Phillip M. Feldman wrote: I currently have a function that uses a list internally but then returns the list items as separate return values as follows: if len(result)==1: return result[0] if len(result)==2: return result[0], result[1] (and so on). Is there a cleaner way to accomplish the same thing? How about using yield and then iterate over the answer: def some_fun(): \tfor result in some_loopable_stuff: \t\t yield result Then call it thusly: for i in some_fun() result = i -Jack (PS, sorry to the OP, you will get two of these -- I forgot to CC the list) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Paul Rubin wrote: Ethan Furman writes: some_list = [1, 2] this, that = func(alist) At least, in 2.5.4 this works. :-) But that fails if there are fewer than two elements in the list. It's better to just make the logic either expect a list, or if it's implementing something like an optional value, code it up explicitly. You may even want to return two lists, the second one possibly empty. It also fails if there are more than two elements in the list, as the rest of my post went on to say. I myself would generally not use such a structure, but that doesn't mean the OP doesn't have a good use case for it. Don't forget, his original question indicated that there could be more than two return elements also. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Ethan Furman writes: > some_list = [1, 2] > this, that = func(alist) > > At least, in 2.5.4 this works. :-) But that fails if there are fewer than two elements in the list. It's better to just make the logic either expect a list, or if it's implementing something like an optional value, code it up explicitly. You may even want to return two lists, the second one possibly empty. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Dr. Phillip M. Feldman wrote: I currently have a function that uses a list internally but then returns the list items as separate return values as follows: if len(result)==1: return result[0] if len(result)==2: return result[0], result[1] (and so on). Is there a cleaner way to accomplish the same thing? To elaborate on Paul's answer, returning the list will also unpack it if you have it set up that way. E.g. def func(alist): return alist some_list = [1, 2] this, that = func(alist) At least, in 2.5.4 this works. :-) Mind you, if you don't have the correct number of return names to match the unpacking you'll get the normal errors from that. Hope this helps! ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
On Thu, Oct 8, 2009 at 12:41 PM, Dr. Phillip M. Feldman wrote: > > I currently have a function that uses a list internally but then returns the > list items as separate return > values as follows: > > if len(result)==1: return result[0] > if len(result)==2: return result[0], result[1] > > (and so on). Is there a cleaner way to accomplish the same thing? It kind of depends on how the caller of your function handles the return values. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Dr. Phillip M. Feldman wrote: I currently have a function that uses a list internally but then returns the list items as separate return values as follows: if len(result)==1: return result[0] if len(result)==2: return result[0], result[1] (and so on). Is there a cleaner way to accomplish the same thing? return tuple(result) But you down want to do that, cause the caller will have a hell of a job getting your result. You may want to simply return the list itself. JM -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
Dr. Phillip M. Feldman schrieb: > I currently have a function that uses a list internally but then returns the > list items as separate return > values as follows: > > if len(result)==1: return result[0] > if len(result)==2: return result[0], result[1] > > (and so on). Is there a cleaner way to accomplish the same thing? You can simply "return result". If you want to make sure that you return a copy of the internal list, do "return list(result)" or "return tuple(result)". Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to code variable number of return arguments?
"Dr. Phillip M. Feldman" writes: > if len(result)==1: return result[0] > if len(result)==2: return result[0], result[1] > > (and so on). Is there a cleaner way to accomplish the same thing? That is poor style. Just return the result as a list. -- http://mail.python.org/mailman/listinfo/python-list
Is there a better way to code variable number of return arguments?
I currently have a function that uses a list internally but then returns the list items as separate return values as follows: if len(result)==1: return result[0] if len(result)==2: return result[0], result[1] (and so on). Is there a cleaner way to accomplish the same thing? -- View this message in context: http://www.nabble.com/Is-there-a-better-way-to-code-variable-number-of-return-arguments--tp25803294p25803294.html Sent from the Python - python-list mailing list archive at Nabble.com. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On Aug 5, 10:46 am, "Martin P. Hellwig" wrote: > Hi List, > > On several occasions I have needed (and build) a parser that reads a > binary piece of data with custom structure. For example (bogus one): > > BE > +-+-+-+-+--++ > | Version | Command | Instruction | Data Length | Data | Filler | > +-+-+-+-+--++ > Version: 6 bits > Command: 4 bits > Instruction: 5 bits > Data Length: 5 bits > Data: 0-31 bits > Filler: filling 0 bits to make the packet dividable by 8 hi, sorry i'm a bit late here, but lepl does exactly this. also, as you asked for in another post, the BitString class allows arbitrary indexing into a sequence of bits. and because it's part of a recursive descent parser you can match "anything" (lepl will handle - although less efficiently - left recursive and ambiguous grammars). see the example at http://www.acooke.org/lepl/binary.html#matching which constructs an ethernet frame and then parses it, extracting the source and destination addresses. feel free to email me with more questions. disclaimer: this is quite new and i don't know of anyone that actually uses it; it is also Python3 only (because it uses bytes()). andrew -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On Thursday 06 August 2009 20:50:30 Martin P. Hellwig wrote: > Thanks all for your insights and suggestions. > It seems to me that there are a couple of ways to this bit manipulation > and a couple of foreign modules to assist you with that. > > Would it be worth the while to do a PEP on this? > Personally I think that it would be nice to have a standard module in > Python for this but perhaps there is a good reason that there isn't > already one. I agree that it would be nice - I think it would be nice if something like this is added to the struct module to give a "this is the obvious way that python handles foreign data" module - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
Thanks all for your insights and suggestions. It seems to me that there are a couple of ways to this bit manipulation and a couple of foreign modules to assist you with that. Would it be worth the while to do a PEP on this? Personally I think that it would be nice to have a standard module in Python for this but perhaps there is a good reason that there isn't already one. -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On Aug 5, 3:46 pm, "Martin P. Hellwig"
wrote:
> Hi List,
>
> On several occasions I have needed (and build) a parser that reads a
> binary piece of data with custom structure. For example (bogus one):
>
> BE
> +-+-+-+-+--++
> | Version | Command | Instruction | Data Length | Data | Filler |
> +-+-+-+-+--++
> Version: 6 bits
> Command: 4 bits
> Instruction: 5 bits
> Data Length: 5 bits
> Data: 0-31 bits
> Filler: filling 0 bits to make the packet dividable by 8
>
> what I usually do is read the packet in binary mode, convert the output
> to a concatenated 'binary string'(i.e. '0101011000110') and then use
> slice indeces to get the right data portions.
> Depending on what I need to do with these portions I convert them to
> whatever is handy (usually an integer).
>
> This works out fine for me. Most of the time I also put the ASCII art
> diagram of this 'protocol' as a comment in the code, making it more
> readable/understandable.
>
> Though there are a couple of things that bothers me with my approach:
> - This seems such a general problem that I think that there must be
> already a general pythonic solution.
> - Using a string for binary representation takes at least 8 times more
> memory for the packet than strictly necessary.
> - Seems to need a lot of prep work before doing the actual parsing.
>
> Any suggestion is greatly appreciated.
>
> --
> MPHhttp://blog.dcuktec.com
> 'If consumed, best digested with added seasoning to own preference.'
Take a look at the bitstring module (in pypi or google code). It's
designed to help make this sort of thing easy and it's more fully
featured than BitVector or BitSet. Internally the data is stored as a
byte array, so memory isn't wasted. It will also do all the dirty work
of bit masking and shifting so that you can concentrate on the real
problems. For example:
>>> s = BitString('0x1232432312') # just to give us some data to play with
>>> ver, comm, instr, bitlen = s.read('uint6, bin4, bin5, uint5')
>>> data = s.readbits(bitlen)
Different interpretations of the binary data are given using Python
properties (e.g. s.hex, s.oct, s.uint, etc.) and it supports bit-wise
slicing, modification, finding, replacing and more. It is also still
in active development (full disclosure: I'm the author :-)).
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On Wednesday 05 August 2009 21:41:26 Martin P. Hellwig wrote: > Yes you are (of course) right, my 'dream' solution would be something > that accepts slice indeces on bit level. Your reasoning did reveal some > flaws in my approach though ;-) This is the first time I have been compared to the sandman... :-) - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On Wednesday 05 August 2009 20:12:05 Paul Rubin wrote: > "Martin P. Hellwig" writes: > > what I usually do is read the packet in binary mode, convert the > > output to a concatenated 'binary string'(i.e. '0101011000110') and > > Something wrong with reading the data words as an integer and using > old fashioned shifts and masks to get at the bit fields? There is nothing wrong with that, and in the final analysis, it is what is used - however, I can sympathize with the OP, as it is much nicer to just call a named thing than to juggle a bunch of hard coded shifts and masks. I think the real point here is sort of meta programming - what is needed is a way to describe and name the elements of the structure so that useful python objects are created. The code I posted for bits does that, and I use it all the time - it is almost as good as being back in 8031 assembler with direct named bit access. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On Wednesday 05 August 2009 16:46:13 Martin P. Hellwig wrote: > Hi List, > > On several occasions I have needed (and build) a parser that reads a > binary piece of data with custom structure. For example (bogus one): > > BE > +-+-+-+-+--++ > > | Version | Command | Instruction | Data Length | Data | Filler | > > +-+-+-+-+--++ > Version: 6 bits > Command: 4 bits > Instruction: 5 bits > Data Length: 5 bits > Data: 0-31 bits > Filler: filling 0 bits to make the packet dividable by 8 > > what I usually do is read the packet in binary mode, convert the output > to a concatenated 'binary string'(i.e. '0101011000110') and then use > slice indeces to get the right data portions. > Depending on what I need to do with these portions I convert them to > whatever is handy (usually an integer). This bit banging stuff is a PITA, no matter what you do. Python does not have bit fields like C. And C bit fields are implementation dependent. Write an extension? Some time ago I asked a similar question, and Castironpi came up with what was essentially an indexed integer, with named bits. It stores the bits natively, but I suspect that the price paid is access time. I enclose a module that you can adapt. It talks about bytes but they are integers really. It is different from what you are doing, as it was aimed at reading and writing bits in a hardware context. If you get your head around the concept, then it may give you some ideas. It should be possible to extend the concept to pass name,length tuples at construction time instead of just a name with an implied length of one bit, and it may make sense to change the underlying type from an integer to an array of one byte integers. It is nice to be able to say: val = bitname() to read, and bitname(1) or bitname(0) to write. I can also write: outputbits[3] = 1 or val = inputbits[5] If you can successfully generalise it to field names It should be very useful. I cannot think of a way though, to not have the "in" and "out" split, but you can program your way around that - you do not have to update "in place". - Hendrik bits.py Description: application/python -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: Parsing Binary Structures; Is there a better way / What is your way?
Paul Rubin wrote: "Martin P. Hellwig" writes: Is there an advantage using shifts and masks over my kitchen type solution? Weren't you complaining about the 8-to-1 expansion from turning each bit to an ascii char? One warning to Martin: If you want your code portable across systems, watch out for big-endian/little-endian issues, as well as alignment ones. Shift & mask code tends to be highly specific to a particular endian-ness, especially if trying to get multiple bits that cross a byte or word boundary. Over the years, I know I've seen at least three endian versions for the same 32bit word. Something like abcd, dcba, and cdab. One advantage of converting first to bitstrings, is that there's just the two places to fix, for portability, the conversion from byte array to bitstring, and the conversion back. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
En Wed, 05 Aug 2009 11:46:13 -0300, Martin P. Hellwig escribió: On several occasions I have needed (and build) a parser that reads a binary piece of data with custom structure. For example (bogus one): BE +-+-+-+-+--++ | Version | Command | Instruction | Data Length | Data | Filler | +-+-+-+-+--++ Version: 6 bits Command: 4 bits Instruction: 5 bits Data Length: 5 bits Data: 0-31 bits Filler: filling 0 bits to make the packet dividable by 8 - Using a string for binary representation takes at least 8 times more memory for the packet than strictly necessary. The size difference isn't so big; an integer takes 12 bytes and a string takes 24+len bytes. "Data" above would take 56 bytes max when stored as a string '0110001...' vs. 12 bytes when using a plain integer (sizes computed on Windows 32bits). Plus, doing bitwise operations in Python isn't a fast operation as it is in C, by example -- so your current implementation might be a quite good one (in pure Python, I mean). -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
"Martin P. Hellwig" writes: > Is there an advantage using shifts and masks over my kitchen type solution? Weren't you complaining about the 8-to-1 expansion from turning each bit to an ascii char? -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On 5 Aug, 20:41, "Martin P. Hellwig" wrote: > Paul Rubin wrote: > > "Martin P. Hellwig" writes: > >> Is there an advantage using shifts and masks over my kitchen type solution? > > > Weren't you complaining about the 8-to-1 expansion from turning each bit > > to an ascii char? > > Yes you are (of course) right, my 'dream' solution would be something > that accepts slice indeces on bit level. Your reasoning did reveal some > flaws in my approach though ;-) > > -- > MPHhttp://blog.dcuktec.com > 'If consumed, best digested with added seasoning to own preference.' Ahh huh -- found it... ID: DLT119 Desc: datasyzygy misc. development, production and accounts. Date: 2008-12-09 File: pybits.py """ Implements a python level interface to reading bit fields with an associated type. Fields are accessed via __getitem__ and must be a valid python name. eg: class MyStructure: a = BitField(4, int) b = BitField(6, int) c = BitField(2, int) d = BitField(36, str) """ Ummm, seems I left the rest pretty vague, but I'm guessing that's what you're after, and I'm fairly curious to see what I wrote... mind you, not too hard to write again now that's jogged my memory! (although after a couple at the pub, not going to do it tonight!) Cheers, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
Paul Rubin wrote: "Martin P. Hellwig" writes: Is there an advantage using shifts and masks over my kitchen type solution? Weren't you complaining about the 8-to-1 expansion from turning each bit to an ascii char? Yes you are (of course) right, my 'dream' solution would be something that accepts slice indeces on bit level. Your reasoning did reveal some flaws in my approach though ;-) -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
Paul Rubin wrote: "Martin P. Hellwig" writes: what I usually do is read the packet in binary mode, convert the output to a concatenated 'binary string'(i.e. '0101011000110') and Something wrong with reading the data words as an integer and using old fashioned shifts and masks to get at the bit fields? No not at all of course, I just found it more readable to slice it instead of shifting and masking; i.e. if the ASCII art schematics of the protocol are included as a comment it is almost an 1 to 1 implementation. Is there an advantage using shifts and masks over my kitchen type solution? -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
"Martin P. Hellwig" writes: > what I usually do is read the packet in binary mode, convert the > output to a concatenated 'binary string'(i.e. '0101011000110') and Something wrong with reading the data words as an integer and using old fashioned shifts and masks to get at the bit fields? -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On 5 Aug 2009, at 19:17 , Bearophile wrote: Have you tried Hachoir? (I think its name may be changed to Fusil, I don't know). Name hasn't been changed (I think fusil is a subproject, something like that) on the other hand the hachoir.org site is dead. But apparently Hachoir was moved to bitbucket (http://bitbucket.org/haypo/hachoir/wiki/Home ) just in time so… -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On 5 Aug 2009, at 16:46 , Martin P. Hellwig wrote: Hi List, On several occasions I have needed (and build) a parser that reads a binary piece of data with custom structure. For example (bogus one): BE +-+-+-+-+--++ | Version | Command | Instruction | Data Length | Data | Filler | +-+-+-+-+--++ Version: 6 bits Command: 4 bits Instruction: 5 bits Data Length: 5 bits Data: 0-31 bits Filler: filling 0 bits to make the packet dividable by 8 what I usually do is read the packet in binary mode, convert the output to a concatenated 'binary string'(i.e. '0101011000110') and then use slice indeces to get the right data portions. Depending on what I need to do with these portions I convert them to whatever is handy (usually an integer). This works out fine for me. Most of the time I also put the ASCII art diagram of this 'protocol' as a comment in the code, making it more readable/understandable. Though there are a couple of things that bothers me with my approach: - This seems such a general problem that I think that there must be already a general pythonic solution. - Using a string for binary representation takes at least 8 times more memory for the packet than strictly necessary. - Seems to need a lot of prep work before doing the actual parsing. Any suggestion is greatly appreciated. The gold standard for binary parsing (and serialization) is probably Erlang's bit syntax, but as far as Python goes you might be interested by Hachoir (http://hachoir.org/ but it seems down right now). It's not going to match your second point, but it can probably help with the rest (caveat: I haven't used hachoir personally). -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
Martin P. Hellwig: > On several occasions I have needed (and build) a parser that reads a > binary piece of data with custom structure. For example (bogus one): > > BE > +-+-+-+-+--++ > | Version | Command | Instruction | Data Length | Data | Filler | > +-+-+-+-+--++ > Version: 6 bits > Command: 4 bits > Instruction: 5 bits > Data Length: 5 bits > Data: 0-31 bits > Filler: filling 0 bits to make the packet dividable by 8 Have you tried Hachoir? (I think its name may be changed to Fusil, I don't know). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
Jon Clements wrote:
IIRC (and I have my doubts) the BitVector module may be of use, but
it's been about 3 years since I had to look at it. I think it used the
C equiv. of short ints to do its work. Otherwise, maybe the array
module, the struct module or even possibly ctypes.
Not much use, but might give a few pointers.
Jon.
I tried struct before (it seemed logical) but it didn't really do what I
expected:
>>> import struct
>>> struct.unpack('?','x')
(True,)
I expected a list like (False, True, True, True, True, False, False,
False). But it is (actually I hope) just an error of my side, so if
somebody knows how to achieve what I want, with this module, please
enlighten me :-)
The array module doesn't seem to have an option for boolean values.
BitVector does look very promising, but after peeking in the source it
does more or less the same as what I do with converting back and forth
of 'bitstrings'.
I haven't thought about using ctypes for this, excellent suggestion!
Thank you for your feedback.
--
MPH
http://blog.dcuktec.com
'If consumed, best digested with added seasoning to own preference.'
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Binary Structures; Is there a better way / What is your way?
On 5 Aug, 15:46, "Martin P. Hellwig" wrote: > Hi List, > > On several occasions I have needed (and build) a parser that reads a > binary piece of data with custom structure. For example (bogus one): > > BE > +-+-+-+-+--++ > | Version | Command | Instruction | Data Length | Data | Filler | > +-+-+-+-+--++ > Version: 6 bits > Command: 4 bits > Instruction: 5 bits > Data Length: 5 bits > Data: 0-31 bits > Filler: filling 0 bits to make the packet dividable by 8 > > what I usually do is read the packet in binary mode, convert the output > to a concatenated 'binary string'(i.e. '0101011000110') and then use > slice indeces to get the right data portions. > Depending on what I need to do with these portions I convert them to > whatever is handy (usually an integer). > > This works out fine for me. Most of the time I also put the ASCII art > diagram of this 'protocol' as a comment in the code, making it more > readable/understandable. > > Though there are a couple of things that bothers me with my approach: > - This seems such a general problem that I think that there must be > already a general pythonic solution. > - Using a string for binary representation takes at least 8 times more > memory for the packet than strictly necessary. > - Seems to need a lot of prep work before doing the actual parsing. > > Any suggestion is greatly appreciated. > > -- > MPHhttp://blog.dcuktec.com > 'If consumed, best digested with added seasoning to own preference.' IIRC (and I have my doubts) the BitVector module may be of use, but it's been about 3 years since I had to look at it. I think it used the C equiv. of short ints to do its work. Otherwise, maybe the array module, the struct module or even possibly ctypes. Not much use, but might give a few pointers. Jon. -- http://mail.python.org/mailman/listinfo/python-list
Parsing Binary Structures; Is there a better way / What is your way?
Hi List, On several occasions I have needed (and build) a parser that reads a binary piece of data with custom structure. For example (bogus one): BE +-+-+-+-+--++ | Version | Command | Instruction | Data Length | Data | Filler | +-+-+-+-+--++ Version: 6 bits Command: 4 bits Instruction: 5 bits Data Length: 5 bits Data: 0-31 bits Filler: filling 0 bits to make the packet dividable by 8 what I usually do is read the packet in binary mode, convert the output to a concatenated 'binary string'(i.e. '0101011000110') and then use slice indeces to get the right data portions. Depending on what I need to do with these portions I convert them to whatever is handy (usually an integer). This works out fine for me. Most of the time I also put the ASCII art diagram of this 'protocol' as a comment in the code, making it more readable/understandable. Though there are a couple of things that bothers me with my approach: - This seems such a general problem that I think that there must be already a general pythonic solution. - Using a string for binary representation takes at least 8 times more memory for the packet than strictly necessary. - Seems to need a lot of prep work before doing the actual parsing. Any suggestion is greatly appreciated. -- MPH http://blog.dcuktec.com 'If consumed, best digested with added seasoning to own preference.' -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On Wed, 20 May 2009 17:08:08 +0100, walterbyrd wrote: I am processing a huge spreadsheet which I have converted to a csv format. Each row will be a wiki page with several sub-headings. The spreadsheet contains information about servers. Wiki sub-headings may include: 'hardware', 'software', 'users', 'network swith settings'. 'Hardware' may include the spreadsheet columns: 'memory', 'cpu', and so on. So the first six columns in the spreadsheet may go under 'hardware' the next six under 'software' and so on. I have already created the wiki pages, using a method similar to what I first posted. But, it seems like there should be a better way to to do it. So, for future reference, I was just wondering. Given that you're already making presumptions about the nature of your data, named constants or enums are the most concise thing to use together with a quick check of the column header row to make sure that the constants really do refer to the right columns. If you want something a little more bullet-proof, create a dictionary mapping the column headers (as read in) to column numbers and use that to generate the slice limits. Since that still relies on the column headers being what you expect them to be, and at least partially in the order you expect them to be, it's probably not enough of a win to bother with. Trying to slice a list by value is never going to look pretty because lists aren't designed to be indexed by value. Worse, if you were doing this a lot (as you imply) then it's going to be horribly inefficient, since you're doing an extra two (or more) O(n) searches for every row. -- Rhodri James *-* Wildebeeste Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On May 19, 5:31 pm, Ben Finney wrote: > That's just the same micro-goal re-stated. What is your larger problem > of which this is a part? Perhaps a better approach can be suggested when > that context is known. I am processing a huge spreadsheet which I have converted to a csv format. Each row will be a wiki page with several sub-headings. The spreadsheet contains information about servers. Wiki sub-headings may include: 'hardware', 'software', 'users', 'network swith settings'. 'Hardware' may include the spreadsheet columns: 'memory', 'cpu', and so on. So the first six columns in the spreadsheet may go under 'hardware' the next six under 'software' and so on. I have already created the wiki pages, using a method similar to what I first posted. But, it seems like there should be a better way to to do it. So, for future reference, I was just wondering. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
> walterbyrd (w) wrote:
>w> On May 8, 5:55 pm, John Yeung wrote:
>>> On May 8, 3:03 pm,walterbyrd wrote:
>>>
>>> > This works, but it seems like there should be a better way.
>>>
>>> > --
>>> > week = ['sun','mon','tue','wed','thu','fri','sat']
>>> > for day in week[week.index('tue'):week.index('fri')]:
>>> > print day
>>> > ---
>>>
>>> I think you should provide much more information, primarily why you
>>> want to do this. What is the larger goal you are trying to achieve?
>w> I am just looking for a less verbose, more elegant, way to print a
>w> slice of a list. What is hard to understand about that? I am not sure
>w> how enumerated types help.
You didn't say that in the OP.
But you can extend the list type to accept slices with strings in them.
The language spec says they should be ints but apparently this is not
enforced. Of course this makes it vulnerable for future misbehaviour.
class KeyList(list):
def __getitem__(self, indx):
if isinstance(indx, slice):
start = indx.start
stop = indx.stop
# add support for step if you want
if not isinstance(start, int):
start = self.index(start)
if not isinstance(stop, int):
stop = self.index(stop)
return list.__getitem__(self, slice(start,stop))
return list.__getitem__(self, indx)
week = KeyList(['sun','mon','tue','wed','thu','fri','sat'])
for day in week['tue':'fri']:
print day
tue
wed
thu
Note that 'fri' is not included according to standard Python conventions
about the end of a slice. Change the code if you are not happy with it
and you don't mind getting inconsistent semantics.
--
Piet van Oostrum
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On Tue, 19 May 2009 14:38:19 -0700, walterbyrd wrote:
> On May 8, 5:55 pm, John Yeung wrote:
>> On May 8, 3:03 pm,walterbyrd wrote:
>>
>> > This works, but it seems like there should be a better way.
>>
>> > --
>> > week = ['sun','mon','tue','wed','thu','fri','sat'] for day in
>> > week[week.index('tue'):week.index('fri')]:
>> > print day
>> > ---
>>
>> I think you should provide much more information, primarily why you
>> want to do this. What is the larger goal you are trying to achieve?
>
> I am just looking for a less verbose, more elegant, way to print a slice
> of a list. What is hard to understand about that? I am not sure how
> enumerated types help.
Printing a slice of a list is about as concise and elegant as possible:
print alist[slice_obj]
or
print alist[start:end:step]
But that's not what the example in your first post suggests. Your example
suggests you have *two* problems:
(1) Given a slice, how to print each item in the slice _individually_.
The answer to that is
for x in aslice:
print x
Pretty concise and elegant.
(2) Given an arbitrary starting and ending _item_ rather than _position_,
how to concisely and elegantly generate a slice.
There are many answers, depending on _why_ you want to do this. One
possible answer is to write a function to do it:
def print_slice(alist, start_item, end_item):
start_pos = alist.index(start_item)
end_pos = alist.index(end_item)
for x in alist[start_pos:end_pos]:
print x
Now print_slice(week, 'tue', 'fri') is pretty concise and elegant.
Another answer is: Don't do that, do something else. If you have an
enumerated type, then you could (in principle) do this:
week = enumerated('mon tue wed thu fri sat sun')
for x in week.tue-week.fri:
print x
depending on the enumerated type itself naturally.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On May 20, 7:38 am, walterbyrd wrote: > On May 8, 5:55 pm, John Yeung wrote: > > > On May 8, 3:03 pm,walterbyrd wrote: > > > > This works, but it seems like there should be a better way. > > > > -- > > > --- > > > I think you should provide much more information, primarily why you > > want to do this. What is the larger goal you are trying to achieve? > > I am just looking for a less verbose, more elegant, way to print a > slice of a list. What is hard to understand about that? Ummm two things, (1) You didn't say that was what you wanted (2) It's a nonsense anyway: Your original statement "choose a slice of alist": answer = alist [lo:hi] Your current statement "print a slice of a list" (one element per line as per your example): can not be done much less verbosely and more elegantly than: for x in alist[lo:hi]: print x Your real problem appears to be the horrid method of deriving lo and hi. You gave ONE example without stating anything more precise than that it was an example of a slice of a list [which was obvious anyway] and didn't specify in what sense of "better" you wanted a better way. So people have to guess what you really want. Guessing that the 'tue' and 'fri' in your one example will always be constants, here are two options: E.g. given week = ['sun', 'mon', 'tue', 'wed', 'thu', 'fri', 'sat'] Option (1): SUN, MON, TUE, WED, THU, FRI, SAT = range(7) for day in week[TUE:FRI]: print day Option (2): for day in week[2:5]: print day HTH, John -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
walterbyrd writes: > On May 8, 5:55 pm, John Yeung wrote: > > On May 8, 3:03 pm,walterbyrd wrote: > > I think you should provide much more information, primarily why you > > want to do this. What is the larger goal you are trying to achieve? > > I am just looking for a less verbose, more elegant, way to print a > slice of a list. That's just the same micro-goal re-stated. What is your larger problem of which this is a part? Perhaps a better approach can be suggested when that context is known. -- \ “What's another word for Thesaurus?” —Steven Wright | `\ | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On Tue, 19 May 2009 22:38:19 +0100, walterbyrd
wrote:
On May 8, 5:55 pm, John Yeung wrote:
On May 8, 3:03 pm,walterbyrd wrote:
> This works, but it seems like there should be a better way.
> --
> week = ['sun','mon','tue','wed','thu','fri','sat']
> for day in week[week.index('tue'):week.index('fri')]:
> print day
> ---
I think you should provide much more information, primarily why you
want to do this. What is the larger goal you are trying to achieve?
I am just looking for a less verbose, more elegant, way to print a
slice of a list. What is hard to understand about that? I am not sure
how enumerated types help.
This is verbose and inelegant because of the way you're storing and
using the data, hence (I presume) John's question. The more elegant
approach is not to try to index a list with strings, but to keep you
"day" data in numeric form and use that to slice with, and for that
enums will greatly help you keep things clear. However, whether that's
worth doing or not depends on the bigger picture, and you haven't
told us anything that would help us figure that out.
--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
walterbyrd wrote:
On May 8, 5:55 pm, John Yeung wrote:
On May 8, 3:03 pm,walterbyrd wrote:
This works, but it seems like there should be a better way.
--
week = ['sun','mon','tue','wed','thu','fri','sat']
for day in week[week.index('tue'):week.index('fri')]:
print day
---
I think you should provide much more information, primarily why you
want to do this. What is the larger goal you are trying to achieve?
I am just looking for a less verbose, more elegant, way to print a
slice of a list.
week[2:5] # ;-)
If you want the interpreter to turn non-ints into ints for you, you will
have to give it some sort of function or mapping to use.
dayint = {day:i for i,day in enumeratr(week)} # works in Py3 at least
week[dayint['tue']:dayint['fri']]
# or, untested, basing attrodic on memory of posted code
class attrodic(): #py3
def __init__(self, dic):
self.__dict__.update(dic)
di = attrodic(week)
week[di.tue:di.fri]
Most elegant, and most setup work ;-).
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On May 8, 5:55 pm, John Yeung wrote:
> On May 8, 3:03 pm,walterbyrd wrote:
>
> > This works, but it seems like there should be a better way.
>
> > --
> > week = ['sun','mon','tue','wed','thu','fri','sat']
> > for day in week[week.index('tue'):week.index('fri')]:
> > print day
> > ---
>
> I think you should provide much more information, primarily why you
> want to do this. What is the larger goal you are trying to achieve?
I am just looking for a less verbose, more elegant, way to print a
slice of a list. What is hard to understand about that? I am not sure
how enumerated types help.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On May 8, 3:03 pm, walterbyrd wrote:
> This works, but it seems like there should be a better way.
>
> --
> week = ['sun','mon','tue','wed','thu','fri','sat']
> for day in week[week.index('tue'):week.index('fri')]:
> print day
> ---
I think you should provide much more information, primarily why you
want to do this. What is the larger goal you are trying to achieve?
In the absence of further information, it seems to me that you are
trying to create an enumerated type. For various ideas to achieve
this simply, depending on your purpose, see
http://norvig.com/python-iaq.html
If you want a more thorough treatment, maybe try this package:
http://pypi.python.org/pypi/enum/
There may be other recipes and packages; you can Google for them using
keywords "python enum" or similar.
John
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On May 8, 3:03 pm, walterbyrd wrote:
> This works, but it seems like there should be a better way.
>
> --
> week = ['sun','mon','tue','wed','thu','fri','sat']
> for day in week[week.index('tue'):week.index('fri')]:
> print day
> ---
Depending on the context this style might help:
>>> week = ['sun','mon','tue','wed','thu','fri','sat']
>>> tue_fri = slice(week.index('tue'), week.index('fri'))
>>> for day in week[tue_fri]:
print day
But I don't really see it as an improvement unless you are using those
intervals repeatedly.
--
http://mail.python.org/mailman/listinfo/python-list
Is there a better way to chose a slice of a list?
This works, but it seems like there should be a better way.
--
week = ['sun','mon','tue','wed','thu','fri','sat']
for day in week[week.index('tue'):week.index('fri')]:
print day
---
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
On Sun, 2009-03-08 at 15:49 +1100, Steven D'Aprano wrote: > If the environmental costs of recycling something are worse than the > environmental costs of throwing it away and making a new one, then > recycling that object is actually harmful. But I digress. Unless you live in a country that imports most of these goods, in which case by recycling you keep money in the economy rather than buying goods from elsewhere - it's never mentioned, but I'm fairly certain that's one of the main reasons that the UK government loves forcing us to recycle so much. (obviously doesn't change how "environmentally harmful" something is) Tim Wintle -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Gabriel Genellina wrote: > Imagine you're working with someone side by side. You write a note in a > piece of paper, put it into an envelope, and hand it to your co-worker. He > opens the envelope, throws it away, takes the note and files it inside a > folder right at the end. And you do this over and over. What's wrong in > this story? > > Please save our trees! Don't waste so many envelopes Nice story, but the moral "conserve what you use" is not always good advice. Bits are not envelopes -- sometimes it is more environmentally friendly to throw them away and create new ones. Consider: mylist[:] = [x for x in mylist if not condition(x)] versus: for i in xrange(len(mylist)-1, -1, -1): x = mylist[i] if condition(x): del mylist[i] The first "wastefully" creates a new list, and the second tries to recycle bits by deleting the items in place. Unless mylist is so huge that your computer starts thrashing trying to make two copies in memory, the first is not only simpler to write and understand, but almost certainly much, much faster than the second. That's not the case in this specific example, but as a general principle, it's worth remembering that it's often better to be wasteful with temporary objects than to use miserly algorithms invented for machines with 64K of memory. (The same lessons can apply for re-world considerations as well. Recycling doesn't just happen, it requires energy and water and other costs. If the environmental costs of recycling something are worse than the environmental costs of throwing it away and making a new one, then recycling that object is actually harmful. But I digress.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Lie Ryan wrote: > mattia wrote: >> Yes, sorry, I have to recycle! But how about this: > rw = [[2,4], [4,5,6],[5,5]] > rw += [[1,1]]*2 > rw >> [[2, 4], [4, 5, 6], [5, 5], [1, 1], [1, 1]] >> How can I recicle in this way using append? > > Not .append() but .extend() Whether you use items += [item]*N or items.extend([item]*N) is mostly a matter of style. You can avoid the intermediate list with items.extend(itertools.repeat(item, N)) but I don't think this approach is faster. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
mattia wrote: Il Sat, 07 Mar 2009 00:05:53 -0200, Gabriel Genellina ha scritto: En Fri, 06 Mar 2009 21:31:01 -0200, mattia escribió: Thanks, I've found another solution here: http://www.obitko.com/tutorials/ genetic-algorithms/selection.php so here is my implementation: def get_fap(fitness, population): fap = [] total = 0 for x in population: f = fitness(x) fap += [(f, x)] total += f return sorted(fap, reverse=True), total Imagine you're working with someone side by side. You write a note in a piece of paper, put it into an envelope, and hand it to your co-worker. He opens the envelope, throws it away, takes the note and files it inside a folder right at the end. And you do this over and over. What's wrong in this story? Please save our trees! Don't waste so many envelopes - that's just what this line does: fap += [(f, x)] Environmentally friendly Pythoneers avoid using discardable intermediate envelopes: fap.append((f, x)) Please recycle! Yes, sorry, I have to recycle! But how about this: rw = [[2,4], [4,5,6],[5,5]] rw += [[1,1]]*2 rw [[2, 4], [4, 5, 6], [5, 5], [1, 1], [1, 1]] rw = [[2,4], [4,5,6],[5,5]] rw.append([1,1]*2) rw [[2, 4], [4, 5, 6], [5, 5], [1, 1, 1, 1]] rw = [[2,4], [4,5,6],[5,5]] rw.append([[1,1]]*2) rw [[2, 4], [4, 5, 6], [5, 5], [[1, 1], [1, 1]]] How can I recicle in this way using append? Not .append() but .extend() >>> rw = [[2,4], [4,5,6],[5,5]] >>> rw.extend([[1,1]]*2) >>> rw > [[2, 4], [4, 5, 6], [5, 5], [1, 1], [1, 1]] -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Il Sat, 07 Mar 2009 00:05:53 -0200, Gabriel Genellina ha scritto: > En Fri, 06 Mar 2009 21:31:01 -0200, mattia escribió: > >> Thanks, I've found another solution here: >> http://www.obitko.com/tutorials/ >> genetic-algorithms/selection.php >> so here is my implementation: >> >> >> def get_fap(fitness, population): >> fap = [] >> total = 0 >> for x in population: >> f = fitness(x) >> fap += [(f, x)] >> total += f >> return sorted(fap, reverse=True), total > > Imagine you're working with someone side by side. You write a note in a > piece of paper, put it into an envelope, and hand it to your co-worker. > He opens the envelope, throws it away, takes the note and files it > inside a folder right at the end. And you do this over and over. What's > wrong in this story? > > Please save our trees! Don't waste so many envelopes - that's just what > this line does: > > fap += [(f, x)] > > Environmentally friendly Pythoneers avoid using discardable intermediate > envelopes: > > fap.append((f, x)) > > Please recycle! Yes, sorry, I have to recycle! But how about this: >>> rw = [[2,4], [4,5,6],[5,5]] >>> rw += [[1,1]]*2 >>> rw [[2, 4], [4, 5, 6], [5, 5], [1, 1], [1, 1]] >>> rw = [[2,4], [4,5,6],[5,5]] >>> rw.append([1,1]*2) >>> rw [[2, 4], [4, 5, 6], [5, 5], [1, 1, 1, 1]] >>> rw = [[2,4], [4,5,6],[5,5]] >>> rw.append([[1,1]]*2) >>> rw [[2, 4], [4, 5, 6], [5, 5], [[1, 1], [1, 1]]] >>> How can I recicle in this way using append? -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
"Gabriel Genellina" writes: > > for x in population: > > f = fitness(x) > > fap += [(f, x)] > > total += f > > return sorted(fap, reverse=True), total > ... > Environmentally friendly Pythoneers avoid using discardable > intermediate envelopes: > > fap.append((f, x)) I'd probably use: fap = list((fitness(x),x) for x in population) total = sum(x for x,y in fap) return sorted(fap, reverse=True), total -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
En Fri, 06 Mar 2009 21:31:01 -0200, mattia escribió: Thanks, I've found another solution here: http://www.obitko.com/tutorials/ genetic-algorithms/selection.php so here is my implementation: def get_fap(fitness, population): fap = [] total = 0 for x in population: f = fitness(x) fap += [(f, x)] total += f return sorted(fap, reverse=True), total Imagine you're working with someone side by side. You write a note in a piece of paper, put it into an envelope, and hand it to your co-worker. He opens the envelope, throws it away, takes the note and files it inside a folder right at the end. And you do this over and over. What's wrong in this story? Please save our trees! Don't waste so many envelopes - that's just what this line does: fap += [(f, x)] Environmentally friendly Pythoneers avoid using discardable intermediate envelopes: fap.append((f, x)) Please recycle! -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Il Fri, 06 Mar 2009 14:13:47 -0800, Scott David Daniels ha scritto:
> mattia wrote:
>> Here is my last shot, where I get rid of all the old intermediate
>> functions:
>>
>> def selection(fitness, population):
>> lp = len(population)
>> roulette_wheel = []
>> for x in population:
>> roulette_wheel += [x]*fitness(x)
>> selected_population = [[]]*lp
>> selected_population[:2] = sorted(population, key=fitness,
>> reverse=True)[:2]
>> selected_population[2:] = [choice(roulette_wheel) for _ in range
>> (lp-2)]
> Try something like this to choose likely couples:
>
> import random
> import bisect
>
> def choose_pairs(fitness_population, decider=random):
> '''Pick and yield pairs weighted by fitness for crossing.
>
> We assume weighted_population has fitness already calculated.
> decide is a parameter to allow testing. '''
> total = 0
> cumulative = []
> candidates = []
> for fitness, individual in set(fitness_population):
> # calculate total weights, extract real candidates if
> fitness > 0:
> total += fitness
> cumulative.append(total)
> candidates.append(individual)
> assert len(candidates) > 1
> while True:
> # pick a candidate by weight
> c0 = decider.random() * total
> first = bisect.bisect_left(cumulative, c0) if first:
> weighting = cumulative[first] - cumulative[first - 1]
> else:
> weighting = cumulative[0]
> # pick another distinct candidate by fitness c1 = choice =
> decider.random() * (total - weighting) if choice >=
> cumulative[first] - weighting:
> choice += weight # adjust to avoid selecting first
> second = bisect.bisect_left(cumulative, choice) yield
> candidates[first], candidates[second]
>
> --Scott David Daniels
> scott.dani...@acm.org
Thanks, I've found another solution here: http://www.obitko.com/tutorials/
genetic-algorithms/selection.php
so here is my implementation:
def create_chromosome(min, max, length):
return [randint(min, max) for i in range(length)]
def create_population(nelem, min, max, length):
# preconditions: nelem > 1 and nelem is even
if not nelem > 1:
nelem = 2
if not nelem%2 == 0:
print("The population must have an even number of elements.
Correcting...")
nelem += 1
return [create_chromosome(min, max, length) for i in range(nelem)]
def get_fap(fitness, population):
fap = []
total = 0
for x in population:
f = fitness(x)
fap += [(f, x)]
total += f
return sorted(fap, reverse=True), total
def my_rw():
list, tot = get_fap(sum, pop)
r = randint(0, tot-1)
i = 0
print(r)
for f, e in list:
i += f
print(i)
if i > r:
return e
return [] # never reached
if __name__ == "__main__":
pop = create_population(5, 0, 1, 10)
# selection_mat(sum, pop)
#print(rw(sum, pop))
list, tot = get_fap(sum, pop)
print(list)
print(tot)
for i in range(6):
print(my_rw())
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Il Fri, 06 Mar 2009 18:46:44 -0300, andrew cooke ha scritto: > i have not been following this discussion in detail, so someone may have > already explained this, but it should not be necessary to actually > construct the roulette wheel to select values from it. what you are > doing is selecting from a list where the there are different > probabilities of selecting different entries. i am pretty sure that can > be done more efficiently than by constructing a new list with many more > entries whose aim is to simulate that (which is what the roulette wheel > seems to be in your code, if i have understood correctly). > > more precisely, i think you can adapt the trick used to select a line at > random from a file by scanning the file just once. > > sorry if i have misunderstood, > andrew Well, I believe that using the right distribution I can for sure find a better way for doing the roulette wheel selection. When I'll have enough time I'll pick up my statistics book. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
mattia wrote: Here is my last shot, where I get rid of all the old intermediate functions: def selection(fitness, population): lp = len(population) roulette_wheel = [] for x in population: roulette_wheel += [x]*fitness(x) selected_population = [[]]*lp selected_population[:2] = sorted(population, key=fitness, reverse=True)[:2] selected_population[2:] = [choice(roulette_wheel) for _ in range (lp-2)] Try something like this to choose likely couples: import random import bisect def choose_pairs(fitness_population, decider=random): '''Pick and yield pairs weighted by fitness for crossing. We assume weighted_population has fitness already calculated. decide is a parameter to allow testing. ''' total = 0 cumulative = [] candidates = [] for fitness, individual in set(fitness_population): # calculate total weights, extract real candidates if fitness > 0: total += fitness cumulative.append(total) candidates.append(individual) assert len(candidates) > 1 while True: # pick a candidate by weight c0 = decider.random() * total first = bisect.bisect_left(cumulative, c0) if first: weighting = cumulative[first] - cumulative[first - 1] else: weighting = cumulative[0] # pick another distinct candidate by fitness c1 = choice = decider.random() * (total - weighting) if choice >= cumulative[first] - weighting: choice += weight # adjust to avoid selecting first second = bisect.bisect_left(cumulative, choice) yield candidates[first], candidates[second] --Scott David Daniels scott.dani...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
i have not been following this discussion in detail, so someone may have already explained this, but it should not be necessary to actually construct the roulette wheel to select values from it. what you are doing is selecting from a list where the there are different probabilities of selecting different entries. i am pretty sure that can be done more efficiently than by constructing a new list with many more entries whose aim is to simulate that (which is what the roulette wheel seems to be in your code, if i have understood correctly). more precisely, i think you can adapt the trick used to select a line at random from a file by scanning the file just once. sorry if i have misunderstood, andrew -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Il Fri, 06 Mar 2009 22:28:00 +0100, Peter Otten ha scritto: > mattia wrote: > >> Il Fri, 06 Mar 2009 14:06:14 +0100, Peter Otten ha scritto: >> >>> mattia wrote: >>> Hi, I'm new to python, and as the title says, can I improve this snippet (readability, speed, tricks): def get_fitness_and_population(fitness, population): return [(fitness(x), x) for x in population] def selection(fitness, population): ''' Select the parent chromosomes from a population according to their fitness (the better fitness, the bigger chance to be selected) ''' selected_population = [] fap = get_fitness_and_population(fitness, population) pop_len = len(population) # elitism (it prevents a loss of the best found solution) # take the only 2 best solutions elite_population = sorted(fap) selected_population += [elite_population[pop_len-1][1]] + [elite_population[pop_len-2][1]] # go on with the rest of the elements for i in range(pop_len-2): # do something >>> >>> def selection1(fitness, population, N=2): >>> rest = sorted(population, key=fitness, reverse=True) best = >>> rest[:N] del rest[:N] >>> # work with best and rest >>> >>> >>> def selection2(fitness, population, N=2): >>> decorated = [(-fitness(p), p) for p in population] >>> heapq.heapify(decorated) >>> >>> best = [heapq.heappop(decorated)[1] for _ in range(N)] rest = [p >>> for f, p in decorated] >>> # work with best and rest >>> >>> Both implementations assume that you are no longer interested in the >>> individuals' fitness once you have partitioned the population in two >>> groups. >>> >>> In theory the second is more efficient for "small" N and "large" >>> populations. >>> >>> Peter >> >> Ok, but the fact is that I save the best individuals of the current >> population, than I'll have to choose the others elements of the new >> population (than will be N-2) in a random way. The common way is using >> a roulette wheel selection (based on the fitness of the individuals, if >> the total fitness is 200, and one individual has a fitness of 10, that >> this individual will have a 0.05 probability to be selected to form the >> new population). So in the selection of the best solution I have to use >> the fitness in order to get the best individual, the last individual >> use the fitness to have a chance to be selected. Obviously the old >> population anf the new population must have the same number of >> individuals. > > You're right, it was a bad idea. > > Peter Here is my last shot, where I get rid of all the old intermediate functions: def selection(fitness, population): lp = len(population) roulette_wheel = [] for x in population: roulette_wheel += [x]*fitness(x) selected_population = [[]]*lp selected_population[:2] = sorted(population, key=fitness, reverse=True)[:2] selected_population[2:] = [choice(roulette_wheel) for _ in range (lp-2)] -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
mattia wrote: > Il Fri, 06 Mar 2009 14:06:14 +0100, Peter Otten ha scritto: > >> mattia wrote: >> >>> Hi, I'm new to python, and as the title says, can I improve this >>> snippet (readability, speed, tricks): >>> >>> def get_fitness_and_population(fitness, population): >>> return [(fitness(x), x) for x in population] >>> >>> def selection(fitness, population): >>> ''' >>> Select the parent chromosomes from a population according to their >>> fitness (the better fitness, the bigger chance to be selected) ''' >>> selected_population = [] >>> fap = get_fitness_and_population(fitness, population) pop_len = >>> len(population) >>> # elitism (it prevents a loss of the best found solution) # take >>> the only 2 best solutions >>> elite_population = sorted(fap) >>> selected_population += [elite_population[pop_len-1][1]] + >>> [elite_population[pop_len-2][1]] >>> # go on with the rest of the elements for i in range(pop_len-2): >>> # do something >> >> def selection1(fitness, population, N=2): >> rest = sorted(population, key=fitness, reverse=True) best = rest[:N] >> del rest[:N] >> # work with best and rest >> >> >> def selection2(fitness, population, N=2): >> decorated = [(-fitness(p), p) for p in population] >> heapq.heapify(decorated) >> >> best = [heapq.heappop(decorated)[1] for _ in range(N)] rest = [p for >> f, p in decorated] >> # work with best and rest >> >> Both implementations assume that you are no longer interested in the >> individuals' fitness once you have partitioned the population in two >> groups. >> >> In theory the second is more efficient for "small" N and "large" >> populations. >> >> Peter > > Ok, but the fact is that I save the best individuals of the current > population, than I'll have to choose the others elements of the new > population (than will be N-2) in a random way. The common way is using a > roulette wheel selection (based on the fitness of the individuals, if the > total fitness is 200, and one individual has a fitness of 10, that this > individual will have a 0.05 probability to be selected to form the new > population). So in the selection of the best solution I have to use the > fitness in order to get the best individual, the last individual use the > fitness to have a chance to be selected. Obviously the old population anf > the new population must have the same number of individuals. You're right, it was a bad idea. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Il Fri, 06 Mar 2009 14:06:14 +0100, Peter Otten ha scritto: > mattia wrote: > >> Hi, I'm new to python, and as the title says, can I improve this >> snippet (readability, speed, tricks): >> >> def get_fitness_and_population(fitness, population): >> return [(fitness(x), x) for x in population] >> >> def selection(fitness, population): >> ''' >> Select the parent chromosomes from a population according to their >> fitness (the better fitness, the bigger chance to be selected) ''' >> selected_population = [] >> fap = get_fitness_and_population(fitness, population) pop_len = >> len(population) >> # elitism (it prevents a loss of the best found solution) # take >> the only 2 best solutions >> elite_population = sorted(fap) >> selected_population += [elite_population[pop_len-1][1]] + >> [elite_population[pop_len-2][1]] >> # go on with the rest of the elements for i in range(pop_len-2): >> # do something > > def selection1(fitness, population, N=2): > rest = sorted(population, key=fitness, reverse=True) best = rest[:N] > del rest[:N] > # work with best and rest > > > def selection2(fitness, population, N=2): > decorated = [(-fitness(p), p) for p in population] > heapq.heapify(decorated) > > best = [heapq.heappop(decorated)[1] for _ in range(N)] rest = [p for > f, p in decorated] > # work with best and rest > > Both implementations assume that you are no longer interested in the > individuals' fitness once you have partitioned the population in two > groups. > > In theory the second is more efficient for "small" N and "large" > populations. > > Peter Ok, but the fact is that I save the best individuals of the current population, than I'll have to choose the others elements of the new population (than will be N-2) in a random way. The common way is using a roulette wheel selection (based on the fitness of the individuals, if the total fitness is 200, and one individual has a fitness of 10, that this individual will have a 0.05 probability to be selected to form the new population). So in the selection of the best solution I have to use the fitness in order to get the best individual, the last individual use the fitness to have a chance to be selected. Obviously the old population anf the new population must have the same number of individuals. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Chris Rebert writes: >for i in range(len(fap)): >selected_population.append(choice(rw)) "for i in range(len(something))" is a bit of a code smell. You could instead say: selected_population.extend(choice(rw) for x in fap) The unused "x" is also a slight code smell, but the most obvious cures involve using the itertools module in ways that are worse than the disease. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
mattia wrote: > Hi, I'm new to python, and as the title says, can I improve this snippet > (readability, speed, tricks): > > def get_fitness_and_population(fitness, population): > return [(fitness(x), x) for x in population] > > def selection(fitness, population): > ''' > Select the parent chromosomes from a population according to their > fitness (the better fitness, the bigger chance to be selected) > ''' > selected_population = [] > fap = get_fitness_and_population(fitness, population) > pop_len = len(population) > # elitism (it prevents a loss of the best found solution) > # take the only 2 best solutions > elite_population = sorted(fap) > selected_population += [elite_population[pop_len-1][1]] + > [elite_population[pop_len-2][1]] > # go on with the rest of the elements > for i in range(pop_len-2): > # do something def selection1(fitness, population, N=2): rest = sorted(population, key=fitness, reverse=True) best = rest[:N] del rest[:N] # work with best and rest def selection2(fitness, population, N=2): decorated = [(-fitness(p), p) for p in population] heapq.heapify(decorated) best = [heapq.heappop(decorated)[1] for _ in range(N)] rest = [p for f, p in decorated] # work with best and rest Both implementations assume that you are no longer interested in the individuals' fitness once you have partitioned the population in two groups. In theory the second is more efficient for "small" N and "large" populations. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
On Fri, Mar 6, 2009 at 3:52 AM, mattia wrote: > Il Fri, 06 Mar 2009 03:43:22 -0800, Chris Rebert ha scritto: > >> On Fri, Mar 6, 2009 at 3:07 AM, mattia wrote: >>> Great, the for statement has not to deal with fap anymore, but with >>> another sequence, like this: >>> >>> def get_roulette_wheel(weight_value_pairs): >>> roulette_wheel = [] >>> for weight, value in weight_value_pairs: >>> roulette_wheel += [value]*weight >>> return roulette_wheel >>> >>> def selection(fitness, population): >>> ... >>> rw = get_roulette_wheel(fap) >>> for i in range(pop_len-2): >>> selected_population += [choice(rw)] >>> return selected_population >>> >>> I think that using [choice(rw)]*len(fap) will produce the same sequence >>> repeted len(fap) times... >> >> Revision to this new code: >> >> def get_roulette_wheel(weight_value_pairs): >> return [[value]*weight for weight, value in weight_value_pairs] >> >> def selection(fitness, population): >> ... >> rw = get_roulette_wheel(fap) >> for i in range(len(fap)): >> selected_population.append(choice(rw)) >> return selected_population >> >> Cheers, >> Chris > > Great, append is equivalent to += right? or more efficient? Yes. .append(item) is equivalent to += [item] and is more efficient. Cheers, Chris -- I have a blog: http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Il Fri, 06 Mar 2009 03:43:22 -0800, Chris Rebert ha scritto: > On Fri, Mar 6, 2009 at 3:07 AM, mattia wrote: >> Great, the for statement has not to deal with fap anymore, but with >> another sequence, like this: >> >> def get_roulette_wheel(weight_value_pairs): >> roulette_wheel = [] >> for weight, value in weight_value_pairs: >> roulette_wheel += [value]*weight >> return roulette_wheel >> >> def selection(fitness, population): >> ... >> rw = get_roulette_wheel(fap) >> for i in range(pop_len-2): >> selected_population += [choice(rw)] >> return selected_population >> >> I think that using [choice(rw)]*len(fap) will produce the same sequence >> repeted len(fap) times... > > Revision to this new code: > > def get_roulette_wheel(weight_value_pairs): >return [[value]*weight for weight, value in weight_value_pairs] > > def selection(fitness, population): >... >rw = get_roulette_wheel(fap) >for i in range(len(fap)): >selected_population.append(choice(rw)) >return selected_population > > Cheers, > Chris Great, append is equivalent to += right? or more efficient? -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
On Fri, Mar 6, 2009 at 3:07 AM, mattia wrote: > Great, the for statement has not to deal with fap anymore, but with > another sequence, like this: > > def get_roulette_wheel(weight_value_pairs): > roulette_wheel = [] > for weight, value in weight_value_pairs: > roulette_wheel += [value]*weight > return roulette_wheel > > def selection(fitness, population): > ... > rw = get_roulette_wheel(fap) > for i in range(pop_len-2): > selected_population += [choice(rw)] > return selected_population > > I think that using [choice(rw)]*len(fap) will produce the same sequence > repeted len(fap) times... Revision to this new code: def get_roulette_wheel(weight_value_pairs): return [[value]*weight for weight, value in weight_value_pairs] def selection(fitness, population): ... rw = get_roulette_wheel(fap) for i in range(len(fap)): selected_population.append(choice(rw)) return selected_population Cheers, Chris -- I have a blog: http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
Il Fri, 06 Mar 2009 10:19:22 +, mattia ha scritto: > Hi, I'm new to python, and as the title says, can I improve this snippet > (readability, speed, tricks): > > def get_fitness_and_population(fitness, population): > return [(fitness(x), x) for x in population] > > def selection(fitness, population): > ''' > Select the parent chromosomes from a population according to their > fitness (the better fitness, the bigger chance to be selected) ''' > selected_population = [] > fap = get_fitness_and_population(fitness, population) pop_len = > len(population) > # elitism (it prevents a loss of the best found solution) # take the > only 2 best solutions > elite_population = sorted(fap) > selected_population += [elite_population[pop_len-1][1]] + > [elite_population[pop_len-2][1]] > # go on with the rest of the elements for i in range(pop_len-2): > # do something Great, the for statement has not to deal with fap anymore, but with another sequence, like this: def get_roulette_wheel(weight_value_pairs): roulette_wheel = [] for weight, value in weight_value_pairs: roulette_wheel += [value]*weight return roulette_wheel def selection(fitness, population): ... rw = get_roulette_wheel(fap) for i in range(pop_len-2): selected_population += [choice(rw)] return selected_population I think that using [choice(rw)]*len(fap) will produce the same sequence repeted len(fap) times... -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way of doing this?
On Fri, Mar 6, 2009 at 2:19 AM, mattia wrote: > Hi, I'm new to python, and as the title says, can I improve this snippet > (readability, speed, tricks): > > def get_fitness_and_population(fitness, population): > return [(fitness(x), x) for x in population] > > def selection(fitness, population): > ''' > Select the parent chromosomes from a population according to their > fitness (the better fitness, the bigger chance to be selected) > ''' > selected_population = [] > fap = get_fitness_and_population(fitness, population) > pop_len = len(population) > # elitism (it prevents a loss of the best found solution) > # take the only 2 best solutions > elite_population = sorted(fap) > selected_population += [elite_population[pop_len-1][1]] + > [elite_population[pop_len-2][1]] > # go on with the rest of the elements > for i in range(pop_len-2): > # do something Removing the unnecessary use of sorted() and using list.pop() rather than explicit indices: def selection(fitness, population): ''' Select the parent chromosomes from a population according to their fitness (the better fitness, the bigger chance to be selected) ''' fap = get_fitness_and_population(fitness, population) fap.sort() # elitism (it prevents a loss of the best found solution) # take the only 2 best solutions selected_population = [fap.pop()[1] for i in range(2)] # go on with the rest of the elements for fit, pop in fap: #do something Cheers, Chris -- I have a blog: http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Is there a better way of doing this?
Hi, I'm new to python, and as the title says, can I improve this snippet (readability, speed, tricks): def get_fitness_and_population(fitness, population): return [(fitness(x), x) for x in population] def selection(fitness, population): ''' Select the parent chromosomes from a population according to their fitness (the better fitness, the bigger chance to be selected) ''' selected_population = [] fap = get_fitness_and_population(fitness, population) pop_len = len(population) # elitism (it prevents a loss of the best found solution) # take the only 2 best solutions elite_population = sorted(fap) selected_population += [elite_population[pop_len-1][1]] + [elite_population[pop_len-2][1]] # go on with the rest of the elements for i in range(pop_len-2): # do something -- http://mail.python.org/mailman/listinfo/python-list
Re: thanks very much indeed for your help is there a better way to do this (python3) newby
On Feb 24, 6:29 am, "Rhodri James"
wrote:
> On Mon, 23 Feb 2009 23:33:31 -, Gary Wood wrote:
> > '''exercise to complete and test this function'''
> > import string
> > def joinStrings(items):
> > '''Join all the strings in stringList into one string,
> > and return the result. For example:
> > >>> print joinStrings(['very', 'hot', 'day'])
> > 'veryhotday'
> > '''
> > for i in items:
> > return (''.join(items))
>
> As I'm sure your teacher will point out, this is sub-optimal :-)
> That for-loop isn't doing anything, because you always return
> out of it at the first iteration.
>
> I suspect that you're expected to concatenate the strings
> together by hand and return the resulting string once you've
> done them all. Trying writing it that way.
>
> PS: it helps a lot if what you say in the doc string matches
> what you write in the rest of the code. In this case you
> call your input string "items", but then say "Join all the
> strings in *stringList*..."
>
> --
> Rhodri James *-* Wildebeeste Herder to the Masses
Here is another way of doing this:
print ('very' + 'hot' + 'day')
Hope this helps.
--
http://mail.python.org/mailman/listinfo/python-list
