Re: better way for ' '.join(args) + '\n'?
On Saturday, 27 October 2012 03:12:31 UTC+5:30, Tycho Andersen wrote: On Fri, Oct 26, 2012 at 05:36:50PM -0400, Dave Angel wrote: On 10/26/2012 05:26 PM, Tycho Andersen wrote: Assuming it's the length of the list that's the problem, not the length of the strings in the list... args = ['foo', 'bar', 'baz'] args[-1] = args[-1] + '\n' line = ' '.join(args) \t Main problem with that is the trailing space before the newline. If that's not a problem, then fine. What trailing space before the newline? The other solutions have it, the above does not. However, the above does mutate args, which isn't all that great. Alas, if you want the performance of mutable structures, you're probably going to have to mutate something. (In any case, it's easy enough to change it back, though ugly.) Not sure why we try so hard to optimize something that's going to take negligible time. The same reason some people enjoy sporting events: it's fun :-) Me too \t -- http://mail.python.org/mailman/listinfo/python-list
RE: better way for ' '.join(args) + '\n'?
Thomas Rachel wrote:
Am 26.10.2012 09:49 schrieb Ulrich Eckhardt:
Hi!
General advise when assembling strings is to not concatenate them
repeatedly but instead use string's join() function, because it avoids
repeated reallocations and is at least as expressive as any alternative.
What I have now is a case where I'm assembling lines of text for driving
a program with a commandline interface.
Stop.
In this case, you think too complicated.
Just do
subprocess.Popen(['prog', 'foo', 'bar', 'baz'])
- is the most safest thing for this use case.
If it should not be possible for any reason, you should be aware of any
traps you could catch - e.g., if you want to feed your string to a
Bourne shell, you should escape the strings properly.
In such cases, I use
def shellquote(*strs):
rInput: file names, output: ''-enclosed strings where every ' is
replaced with '\''. Intended for usage with the shell.
# just take over everything except ';
# replace ' with '\''
# The shell sees ''' as ''\'''\'''\'''. Ugly, but works.
return .join([
'+st.replace(','\\'')+'
for st in strs
])
so I can use
shellquote('program name', 'argument 1', '$arg 2',
even args containing a ' are ok)
For Windows, you'll have to modify this somehow.
The subprocess module suggests using pipes.quote for escaping.
a
('program name', 'argument 1', '$arg 2', even args containing a ' are ok)
import pipes
map(pipes.quote, a)
['program name', '\'argument 1\'', '$arg 2', '\'even args containing a
\'\'\' are ok\'']
' '.join(a)
'\'program name\' \'argument 1\' \'$arg 2\' \'even args containing a \'\\\'\'
are ok\''
Ramit Prasad
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.
--
http://mail.python.org/mailman/listinfo/python-list
Re: better way for ' '.join(args) + '\n'?
Am 26.10.2012 09:49 schrieb Ulrich Eckhardt:
Hi!
General advise when assembling strings is to not concatenate them
repeatedly but instead use string's join() function, because it avoids
repeated reallocations and is at least as expressive as any alternative.
What I have now is a case where I'm assembling lines of text for driving
a program with a commandline interface.
Stop.
In this case, you think too complicated.
Just do
subprocess.Popen(['prog', 'foo', 'bar', 'baz'])
- is the most safest thing for this use case.
If it should not be possible for any reason, you should be aware of any
traps you could catch - e.g., if you want to feed your string to a
Bourne shell, you should escape the strings properly.
In such cases, I use
def shellquote(*strs):
rInput: file names, output: ''-enclosed strings where every ' is
replaced with '\''. Intended for usage with the shell.
# just take over everything except ';
# replace ' with '\''
# The shell sees ''' as ''\'''\'''\'''. Ugly, but works.
return .join([
'+st.replace(','\\'')+'
for st in strs
])
so I can use
shellquote('program name', 'argument 1', '$arg 2',
even args containing a ' are ok)
For Windows, you'll have to modify this somehow.
HTH,
Thomas
--
http://mail.python.org/mailman/listinfo/python-list
better way for ' '.join(args) + '\n'?
Hi! General advise when assembling strings is to not concatenate them repeatedly but instead use string's join() function, because it avoids repeated reallocations and is at least as expressive as any alternative. What I have now is a case where I'm assembling lines of text for driving a program with a commandline interface. In this scenario, I'm currently doing this: args = ['foo', 'bar', 'baz'] line = ' '.join(args) + '\n' So, in other words, I'm avoiding all the unnecessary copying, just to make another copy to append the final newline. The only way around this that I found involves creating an intermediate sequence like ['foo', ' ', 'bar', ' ', 'baz', '\n']. This can be done rather cleanly with a generator: def helper(s): for i in s[:-1]: yield i yield ' ' yield s[-1] yield '\n' line = ''.join(tmp(args)) Efficiency-wise, this is satisfactory. However, readability counts and that is where this version fails and that is the reason why I'm writing this message. So, dear fellow Pythonistas, any ideas to improve the original versions efficiency while preserving its expressiveness? Oh, for all those that are tempted to tell me that this is not my bottleneck unless it's called in a very tight loop, you're right. Indeed, the overhead of the communication channel TCP between the two programs is by far dwarving the few microseconds I could save here. I'm still interested in learning new and better solutions though. Cheers! Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: better way for ' '.join(args) + '\n'?
Ulrich Eckhardt wrote: Hi! General advise when assembling strings is to not concatenate them repeatedly but instead use string's join() function, because it avoids repeated reallocations and is at least as expressive as any alternative. What I have now is a case where I'm assembling lines of text for driving a program with a commandline interface. In this scenario, I'm currently doing this: args = ['foo', 'bar', 'baz'] line = ' '.join(args) + '\n' So, in other words, I'm avoiding all the unnecessary copying, just to make another copy to append the final newline. The only way around this that I found involves creating an intermediate sequence like ['foo', ' ', 'bar', ' ', 'baz', '\n']. This can be done rather cleanly with a generator: def helper(s): for i in s[:-1]: yield i yield ' ' yield s[-1] yield '\n' line = ''.join(tmp(args)) Efficiency-wise, this is satisfactory. No, it is not. In a quick timeit test it takes 5 to 10 times as long as the original. Remember that function calls are costly, and that with s[:-1] you are trading the extra string for an extra list. Also, you are doubling the loop implicit in str.join() with the explicit one in your oh-so-efficient generator. However, readability counts and that is where this version fails and that is the reason why I'm writing this message. So, dear fellow Pythonistas, any ideas to improve the original versions efficiency while preserving its expressiveness? Oh, for all those that are tempted to tell me that this is not my bottleneck unless it's called in a very tight loop, you're right. Indeed, the overhead of the communication channel TCP between the two programs is by far dwarving the few microseconds I could save here. I'm still interested in learning new and better solutions though. Even if it were the bottleneck the helper generator approach would still be unhelpful. -- http://mail.python.org/mailman/listinfo/python-list
Re: better way for ' '.join(args) + '\n'?
On Fri, 26 Oct 2012 09:49:50 +0200, Ulrich Eckhardt wrote:
Hi!
General advise when assembling strings is to not concatenate them
repeatedly but instead use string's join() function, because it avoids
repeated reallocations and is at least as expressive as any alternative.
What I have now is a case where I'm assembling lines of text for driving
a program with a commandline interface. In this scenario, I'm currently
doing this:
args = ['foo', 'bar', 'baz']
line = ' '.join(args) + '\n'
So, in other words, I'm avoiding all the unnecessary copying, just to
make another copy to append the final newline.
*shrug*
The difference between ' '.join(sequence) and (' '.join(sequence) + '\n')
is, in Big Oh analysis, insignificant. The first case does O(N)
operations, the second does O(N) + O(N) = 2*O(N) operations, which is
still O(N). In effect, the two differ only by an approximately constant
factor.
If you really care, and you don't mind ending your last line with a
space, just append '\n' to the sequence before calling join.
The only way around this that I found involves creating an intermediate
sequence like ['foo', ' ', 'bar', ' ', 'baz', '\n']. This can be done
rather cleanly with a generator:
def helper(s):
for i in s[:-1]:
yield i
yield ' '
yield s[-1]
yield '\n'
line = ''.join(tmp(args))
Efficiency-wise, this is satisfactory.
Have you actually tested this? I would not be the least surprised if
that's actually less efficient than the (' '.join(seq) + '\n') version.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: better way for ' '.join(args) + '\n'?
Hi Ulrich,
is this acceptable?
args = ['foo', 'bar', 'baz']
args.append('\n')
line = ' '.join(args)
Cheers,
Hubert
On 10/26/2012 09:49 AM, Ulrich Eckhardt wrote:
Hi!
General advise when assembling strings is to not concatenate them
repeatedly but instead use string's join() function, because it avoids
repeated reallocations and is at least as expressive as any alternative.
What I have now is a case where I'm assembling lines of text for driving
a program with a commandline interface. In this scenario, I'm currently
doing this:
args = ['foo', 'bar', 'baz']
line = ' '.join(args) + '\n'
So, in other words, I'm avoiding all the unnecessary copying, just to
make another copy to append the final newline.
The only way around this that I found involves creating an intermediate
sequence like ['foo', ' ', 'bar', ' ', 'baz', '\n']. This can be done
rather cleanly with a generator:
def helper(s):
for i in s[:-1]:
yield i
yield ' '
yield s[-1]
yield '\n'
line = ''.join(tmp(args))
Efficiency-wise, this is satisfactory. However, readability counts and
that is where this version fails and that is the reason why I'm writing
this message. So, dear fellow Pythonistas, any ideas to improve the
original versions efficiency while preserving its expressiveness?
Oh, for all those that are tempted to tell me that this is not my
bottleneck unless it's called in a very tight loop, you're right.
Indeed, the overhead of the communication channel TCP between the two
programs is by far dwarving the few microseconds I could save here. I'm
still interested in learning new and better solutions though.
Cheers!
Uli
--
http://mail.python.org/mailman/listinfo/python-list
Re: better way for ' '.join(args) + '\n'?
On Fri, Oct 26, 2012 at 09:49:50AM +0200, Ulrich Eckhardt wrote: Hi! General advise when assembling strings is to not concatenate them repeatedly but instead use string's join() function, because it avoids repeated reallocations and is at least as expressive as any alternative. What I have now is a case where I'm assembling lines of text for driving a program with a commandline interface. In this scenario, I'm currently doing this: args = ['foo', 'bar', 'baz'] line = ' '.join(args) + '\n' Assuming it's the length of the list that's the problem, not the length of the strings in the list... args = ['foo', 'bar', 'baz'] args[-1] = args[-1] + '\n' line = ' '.join(args) \t -- http://mail.python.org/mailman/listinfo/python-list
Re: better way for ' '.join(args) + '\n'?
On 10/26/2012 05:26 PM, Tycho Andersen wrote: On Fri, Oct 26, 2012 at 09:49:50AM +0200, Ulrich Eckhardt wrote: Hi! General advise when assembling strings is to not concatenate them repeatedly but instead use string's join() function, because it avoids repeated reallocations and is at least as expressive as any alternative. What I have now is a case where I'm assembling lines of text for driving a program with a commandline interface. In this scenario, I'm currently doing this: args = ['foo', 'bar', 'baz'] line = ' '.join(args) + '\n' Assuming it's the length of the list that's the problem, not the length of the strings in the list... args = ['foo', 'bar', 'baz'] args[-1] = args[-1] + '\n' line = ' '.join(args) \t Main problem with that is the trailing space before the newline. If that's not a problem, then fine. Not sure why we try so hard to optimize something that's going to take negligible time. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: better way for ' '.join(args) + '\n'?
On Fri, Oct 26, 2012 at 05:36:50PM -0400, Dave Angel wrote: On 10/26/2012 05:26 PM, Tycho Andersen wrote: Assuming it's the length of the list that's the problem, not the length of the strings in the list... args = ['foo', 'bar', 'baz'] args[-1] = args[-1] + '\n' line = ' '.join(args) \t Main problem with that is the trailing space before the newline. If that's not a problem, then fine. What trailing space before the newline? The other solutions have it, the above does not. However, the above does mutate args, which isn't all that great. Alas, if you want the performance of mutable structures, you're probably going to have to mutate something. (In any case, it's easy enough to change it back, though ugly.) Not sure why we try so hard to optimize something that's going to take negligible time. The same reason some people enjoy sporting events: it's fun :-) \t -- http://mail.python.org/mailman/listinfo/python-list
