Re: [Rd] [R] split strings
(diverted to r-devel, a source code patch attached)
Wacek Kusnierczyk wrote:
Allan Engelhardt wrote:
Immaterial, yes, but it is always good to test :) and your solution
*is* faster and it is even faster if you can assume byte strings:
:)
indeed; though if the speed is immaterial (and in this case it
supposedly was), it's probably not worth risking fixed=TRUE removing
'.tif' from the middle of the name, however unlikely this might be (cf
murphy's laws).
but if you can assume that each string ends with a '.tif' (or any other
\..{3} substring), then substr is marginally faster than sub, even as a
three-pass approach, while avoiding the risk of removing '.tif' from the
middle:
strings = sprintf('f:/foo/bar//%s.tif', replicate(1000,
paste(sample(letters, 10), collapse='')))
library(rbenchmark)
benchmark(columns=c('test', 'elapsed'), replications=1000, order=NULL,
substr={basenames=basename(strings); substr(basenames, 1,
nchar(basenames)-4)},
sub=sub('.tif', '', basename(strings), fixed=TRUE, useBytes=TRUE))
# test elapsed
# 1 substr 3.176
# 2sub 3.296
btw., i wonder why negative indices default to 1 in substr:
substr('foobar', -5, 5)
# fooba
# substr('foobar', 1, 5)
substr('foobar', 2, -2)
#
# substr('foobar', 2, 1)
this does not seem to be documented in ?substr. there are ways to make
negative indices meaningful, e.g., by taking them as indexing from
behind (as in, e.g., perl):
# hypothetical
substr('foobar', -5, 5)
# ooba
# substr('foobar', 6-5+1, 5)
substr('foobar', 2, -2)
# ooba
# substr('foobar', 2, 6-2+1)
there is a trivial fix to src/main/character.c that gives substr the
extended functionality -- see the attached patch. the patch has been
created and tested as follows:
svn co https://svn.r-project.org/R/trunk r-devel
cd r-devel
# modifications made to src/main/character.c
svn diff character.c.diff
svn revert -R .
patch -p0 character.c.diff
./configure
make
make check-all
# no problems reported
with the patched substr, the original problem can now be solved more
concisely, using a two-pass approach, with performance still better than
the sub/fixed/bytes one, as follows:
strings = sprintf('f:/foo/bar//%s.tif', replicate(1000,
paste(sample(letters, 10), collapse='')))
library(rbenchmark)
benchmark(columns=c('test', 'elapsed'), replications=1000, order=NULL,
substr=substr(basename(strings), 1, -5),
'substr-nchar'={
basenames=basename(strings)
substr(basenames, 1, nchar(basenames)-4) },
sub=sub('.tif', '', basename(strings), fixed=TRUE, useBytes=TRUE))
# test elapsed
# 1 substr 2.981
# 2 substr-nchar 3.206
# 3 sub 3.273
if this sounds interesting, i can update the docs accordingly.
vQ
Index: src/main/character.c
===
--- src/main/character.c (revision 48667)
+++ src/main/character.c (working copy)
@@ -244,7 +244,12 @@
ss = CHAR(el);
slen = strlen(ss); /* FIXME -- should handle embedded nuls */
buf = R_AllocStringBuffer(slen+1, cbuff);
- if (start 1) start = 1;
+ if (start == 0)
+ start = 1;
+ else if (start 0)
+ start = slen + start + 1;
+ if (stop 0)
+ stop = slen + stop + 1;
if (start stop || start slen) {
buf[0] = '\0';
} else {
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] split strings
Bill Dunlap
TIBCO Software Inc - Spotfire Division
wdunlap tibco.com
-Original Message-
From: r-devel-boun...@r-project.org
[mailto:r-devel-boun...@r-project.org] On Behalf Of Wacek Kusnierczyk
Sent: Thursday, May 28, 2009 5:30 AM
Cc: R help project; r-devel@r-project.org; Allan Engelhardt
Subject: Re: [Rd] [R] split strings
(diverted to r-devel, a source code patch attached)
Wacek Kusnierczyk wrote:
Allan Engelhardt wrote:
Immaterial, yes, but it is always good to test :) and your solution
*is* faster and it is even faster if you can assume byte strings:
:)
indeed; though if the speed is immaterial (and in this case it
supposedly was), it's probably not worth risking fixed=TRUE removing
'.tif' from the middle of the name, however unlikely this
might be (cf
murphy's laws).
but if you can assume that each string ends with a '.tif'
(or any other
\..{3} substring), then substr is marginally faster than
sub, even as a
three-pass approach, while avoiding the risk of removing
'.tif' from the
middle:
strings = sprintf('f:/foo/bar//%s.tif', replicate(1000,
paste(sample(letters, 10), collapse='')))
library(rbenchmark)
benchmark(columns=c('test', 'elapsed'),
replications=1000, order=NULL,
substr={basenames=basename(strings); substr(basenames, 1,
nchar(basenames)-4)},
sub=sub('.tif', '', basename(strings), fixed=TRUE,
useBytes=TRUE))
# test elapsed
# 1 substr 3.176
# 2sub 3.296
btw., i wonder why negative indices default to 1 in substr:
substr('foobar', -5, 5)
# fooba
# substr('foobar', 1, 5)
substr('foobar', 2, -2)
#
# substr('foobar', 2, 1)
this does not seem to be documented in ?substr.
Would your patched code affect the following
use of regexpr's output as input to substr, to
pull out the matched text from the string?
x-c(ooo,good food,bad)
r-regexpr(o+, x)
substring(x,r,attr(r,match.length)+r-1)
[1] ooo oo
substr(x,r,attr(r,match.length)+r-1)
[1] ooo oo
r
[1] 1 2 -1
attr(,match.length)
[1] 3 2 -1
attr(r,match.length)+r-1
[1] 3 3 -3
attr(,match.length)
[1] 3 2 -1
there are
ways to make
negative indices meaningful, e.g., by taking them as indexing from
behind (as in, e.g., perl):
# hypothetical
substr('foobar', -5, 5)
# ooba
# substr('foobar', 6-5+1, 5)
substr('foobar', 2, -2)
# ooba
# substr('foobar', 2, 6-2+1)
there is a trivial fix to src/main/character.c that gives substr the
extended functionality -- see the attached patch. the patch has been
created and tested as follows:
svn co https://svn.r-project.org/R/trunk r-devel
cd r-devel
# modifications made to src/main/character.c
svn diff character.c.diff
svn revert -R .
patch -p0 character.c.diff
./configure
make
make check-all
# no problems reported
with the patched substr, the original problem can now be solved more
concisely, using a two-pass approach, with performance still
better than
the sub/fixed/bytes one, as follows:
strings = sprintf('f:/foo/bar//%s.tif', replicate(1000,
paste(sample(letters, 10), collapse='')))
library(rbenchmark)
benchmark(columns=c('test', 'elapsed'),
replications=1000, order=NULL,
substr=substr(basename(strings), 1, -5),
'substr-nchar'={
basenames=basename(strings)
substr(basenames, 1, nchar(basenames)-4) },
sub=sub('.tif', '', basename(strings), fixed=TRUE,
useBytes=TRUE))
# test elapsed
# 1 substr 2.981
# 2 substr-nchar 3.206
# 3 sub 3.273
if this sounds interesting, i can update the docs accordingly.
vQ
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] split strings
William Dunlap wrote: Would your patched code affect the following use of regexpr's output as input to substr, to pull out the matched text from the string? x-c(ooo,good food,bad) r-regexpr(o+, x) substring(x,r,attr(r,match.length)+r-1) [1] ooo oo no; same output substr(x,r,attr(r,match.length)+r-1) [1] ooo oo no; same output r [1] 1 2 -1 attr(,match.length) [1] 3 2 -1 attr(r,match.length)+r-1 [1] 3 3 -3 attr(,match.length) [1] 3 2 -1 for the positive indices there is no change, as you might expect. if i understand your concern, the issue is that regexpr returns -1 (with the corresponding attribute -1) where there is no match. in this case, you expect as the substring. if there is no match, we have: start = r = -1 (the start you index provide) stop = attr(r) + r - 1 = -1 + -1 -1 = -3 (the stop index you provide) for a string of length n, my patch computes the final indices as follows: start' = n + start - 1 stop' = n + stop - 1 whatever the value of n, stop' - start' = stop - start = -3 - 1 = -4. that is, stop' start', hence an empty string is returned, by virtue of the original code. (see the sources for details.) does this answer your question? vQ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] split strings
Wacek Kusnierczyk wrote: William Dunlap wrote: Would your patched code affect the following use of regexpr's output as input to substr, to pull out the matched text from the string? x-c(ooo,good food,bad) r-regexpr(o+, x) substring(x,r,attr(r,match.length)+r-1) [1] ooo oo no; same output substr(x,r,attr(r,match.length)+r-1) [1] ooo oo no; same output r [1] 1 2 -1 attr(,match.length) [1] 3 2 -1 attr(r,match.length)+r-1 [1] 3 3 -3 attr(,match.length) [1] 3 2 -1 for the positive indices there is no change, as you might expect. if i understand your concern, the issue is that regexpr returns -1 (with the corresponding attribute -1) where there is no match. in this case, you expect as the substring. if there is no match, we have: start = r = -1 (the start you index provide) stop = attr(r) + r - 1 = -1 + -1 -1 = -3 (the stop index you provide) for a string of length n, my patch computes the final indices as follows: start' = n + start - 1 stop' = n + stop - 1 whatever the value of n, stop' - start' = stop - start = -3 - 1 = -4. except for that stop - start = -3 - -1 = -2, but that's still negative, i.e., stop' start'. silly me, sorry. vQ that is, stop' start', hence an empty string is returned, by virtue of the original code. (see the sources for details.) does this answer your question? __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
