Re: [Rcpp-devel] inplace modification more affect other varibles
a and b are the same object: a - seq(1, 0.1, -0.1) b - a pryr::address( a ) [1] 0x7f9504534948 pryr::address( b ) [1] 0x7f9504534948 So clone is what you need here. Implementing copy on write for that kind of example is possible, but would require a lot of additional code, i.e. the iterator would need to handle the write operation. An undesirable side effect of this is that such iterators would be quite less performant, right now Rcpp is close to the metal and uses direct pointers as iterators when it makes sense. A price that everyone would have to pay. no go. Instead, the responsibility is given to the user to clone explicitly when changes will be made to the underlying object. Romain Le 22 oct. 2014 à 04:13, Chenliang Xu luckyr...@gmail.com a écrit : Hi Dirk, Thanks for your quick answer. I don't think Rcpp::clone is what I was looking for. I know `stl_sort_inplace(a)` modify the value of `a`, but it surprise me to see it modify `b`. And it may modify some other variables c, d, e, f..., and it's hard to know which variables point to the same place. On Tue, Oct 21, 2014 at 8:31 PM, Dirk Eddelbuettel e...@debian.org mailto:e...@debian.org wrote: On 21 October 2014 at 20:22, Chenliang Xu wrote: | Hello, | | With the following inplace sorting example, I understand the value of `a` is | sorted inplace, but it's strange to see the value of `b` is also modified. This | can cause some hard to detect bug, since the cpp function may modify a variable | defined in other scope. Very well known issue -- maybe do a search for 'Rcpp::clone' ... In a nutshell, SEXP objects are passed by a __pointer__ and changes do therefore persist. If you want distinct copies, use Rcpp::clone(). Dirk | It seems that rcpp doesn't respect the named field, which is adopted by R to | implement copy-on-modify. I don's see an easy fix on C++ side, since the called | cpp function has no information about variable binding in R. A possible fix is | adding a function `inplace` to R, which ensure the returned variable has named | filed = 0 so is safe to modify inplace. Then, we have to call the function as | `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It | shows clearly that we are breaking the pass-by-value rule in R. | | ```cpp | #include Rcpp.h | using namespace Rcpp; | | // [[Rcpp::export]] | void stl_sort_inplace(NumericVector x) { | std::sort(x.begin(), x.end()); | } | | ``` | | ```r | a - seq(1, 0.1, -0.1) | b - a | # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 | | stl_sort_inplace(a) | | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | b | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | a - seq(1, 0.1, -0.1) | pure_function - function (x) { | y - x | stl_sort_inplace(y) | print(y) | } | pure_function(a) | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | ``` | | ___ | Rcpp-devel mailing list | Rcpp-devel@lists.r-forge.r-project.org mailto:Rcpp-devel@lists.r-forge.r-project.org | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- http://dirk.eddelbuettel.com http://dirk.eddelbuettel.com/ | @eddelbuettel | e...@debian.org mailto:e...@debian.org ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] inplace modification more affect other varibles
Thanks a lot! Does that mean we should never modify an argument passed from R to cpp? On Wed, Oct 22, 2014 at 8:24 AM, Romain Francois rom...@r-enthusiasts.com wrote: a and b are the same object: a - seq(1, 0.1, -0.1) b - a pryr::address( a ) [1] 0x7f9504534948 pryr::address( b ) [1] 0x7f9504534948 So clone is what you need here. Implementing copy on write for that kind of example is possible, but would require a lot of additional code, i.e. the iterator would need to handle the write operation. An undesirable side effect of this is that such iterators would be quite less performant, right now Rcpp is close to the metal and uses direct pointers as iterators when it makes sense. A price that everyone would have to pay. no go. Instead, the responsibility is given to the user to clone explicitly when changes will be made to the underlying object. Romain Le 22 oct. 2014 à 04:13, Chenliang Xu luckyr...@gmail.com a écrit : Hi Dirk, Thanks for your quick answer. I don't think Rcpp::clone is what I was looking for. I know `stl_sort_inplace(a)` modify the value of `a`, but it surprise me to see it modify `b`. And it may modify some other variables c, d, e, f..., and it's hard to know which variables point to the same place. On Tue, Oct 21, 2014 at 8:31 PM, Dirk Eddelbuettel e...@debian.org wrote: On 21 October 2014 at 20:22, Chenliang Xu wrote: | Hello, | | With the following inplace sorting example, I understand the value of `a` is | sorted inplace, but it's strange to see the value of `b` is also modified. This | can cause some hard to detect bug, since the cpp function may modify a variable | defined in other scope. Very well known issue -- maybe do a search for 'Rcpp::clone' ... In a nutshell, SEXP objects are passed by a __pointer__ and changes do therefore persist. If you want distinct copies, use Rcpp::clone(). Dirk | It seems that rcpp doesn't respect the named field, which is adopted by R to | implement copy-on-modify. I don's see an easy fix on C++ side, since the called | cpp function has no information about variable binding in R. A possible fix is | adding a function `inplace` to R, which ensure the returned variable has named | filed = 0 so is safe to modify inplace. Then, we have to call the function as | `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It | shows clearly that we are breaking the pass-by-value rule in R. | | ```cpp | #include Rcpp.h | using namespace Rcpp; | | // [[Rcpp::export]] | void stl_sort_inplace(NumericVector x) { | std::sort(x.begin(), x.end()); | } | | ``` | | ```r | a - seq(1, 0.1, -0.1) | b - a | # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 | | stl_sort_inplace(a) | | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | b | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | a - seq(1, 0.1, -0.1) | pure_function - function (x) { | y - x | stl_sort_inplace(y) | print(y) | } | pure_function(a) | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | ``` | | ___ | Rcpp-devel mailing list | Rcpp-devel@lists.r-forge.r-project.org | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] inplace modification more affect other varibles
Pretty much. But sometimes that's what you want, and Rcpp does not get in the way. You just have to know the rules of the game. BTW, same rules apply when you use .Call C/R API, you are in charge of making the copy when it's needed. Romain Le 22 oct. 2014 à 17:53, Chenliang Xu luckyr...@gmail.com a écrit : Thanks a lot! Does that mean we should never modify an argument passed from R to cpp? On Wed, Oct 22, 2014 at 8:24 AM, Romain Francois rom...@r-enthusiasts.com mailto:rom...@r-enthusiasts.com wrote: a and b are the same object: a - seq(1, 0.1, -0.1) b - a pryr::address( a ) [1] 0x7f9504534948 pryr::address( b ) [1] 0x7f9504534948 So clone is what you need here. Implementing copy on write for that kind of example is possible, but would require a lot of additional code, i.e. the iterator would need to handle the write operation. An undesirable side effect of this is that such iterators would be quite less performant, right now Rcpp is close to the metal and uses direct pointers as iterators when it makes sense. A price that everyone would have to pay. no go. Instead, the responsibility is given to the user to clone explicitly when changes will be made to the underlying object. Romain Le 22 oct. 2014 à 04:13, Chenliang Xu luckyr...@gmail.com mailto:luckyr...@gmail.com a écrit : Hi Dirk, Thanks for your quick answer. I don't think Rcpp::clone is what I was looking for. I know `stl_sort_inplace(a)` modify the value of `a`, but it surprise me to see it modify `b`. And it may modify some other variables c, d, e, f..., and it's hard to know which variables point to the same place. On Tue, Oct 21, 2014 at 8:31 PM, Dirk Eddelbuettel e...@debian.org mailto:e...@debian.org wrote: On 21 October 2014 at 20:22, Chenliang Xu wrote: | Hello, | | With the following inplace sorting example, I understand the value of `a` is | sorted inplace, but it's strange to see the value of `b` is also modified. This | can cause some hard to detect bug, since the cpp function may modify a variable | defined in other scope. Very well known issue -- maybe do a search for 'Rcpp::clone' ... In a nutshell, SEXP objects are passed by a __pointer__ and changes do therefore persist. If you want distinct copies, use Rcpp::clone(). Dirk | It seems that rcpp doesn't respect the named field, which is adopted by R to | implement copy-on-modify. I don's see an easy fix on C++ side, since the called | cpp function has no information about variable binding in R. A possible fix is | adding a function `inplace` to R, which ensure the returned variable has named | filed = 0 so is safe to modify inplace. Then, we have to call the function as | `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It | shows clearly that we are breaking the pass-by-value rule in R. | | ```cpp | #include Rcpp.h | using namespace Rcpp; | | // [[Rcpp::export]] | void stl_sort_inplace(NumericVector x) { | std::sort(x.begin(), x.end()); | } | | ``` | | ```r | a - seq(1, 0.1, -0.1) | b - a | # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 | | stl_sort_inplace(a) | | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | b | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | a - seq(1, 0.1, -0.1) | pure_function - function (x) { | y - x | stl_sort_inplace(y) | print(y) | } | pure_function(a) | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | ``` | | ___ | Rcpp-devel mailing list | Rcpp-devel@lists.r-forge.r-project.org mailto:Rcpp-devel@lists.r-forge.r-project.org | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- http://dirk.eddelbuettel.com http://dirk.eddelbuettel.com/ | @eddelbuettel | e...@debian.org mailto:e...@debian.org ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org mailto:Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] inplace modification more affect other varibles
On Tue, Oct 21, 2014 at 9:22 PM, Chenliang Xu luckyr...@gmail.com wrote: Hello, With the following inplace sorting example, I understand the value of `a` is sorted inplace, but it's strange to see the value of `b` is also modified. This can cause some hard to detect bug, since the cpp function may modify a variable defined in other scope. It seems that rcpp doesn't respect the named field, which is adopted by R to implement copy-on-modify. I don's see an easy fix on C++ side, since the called cpp function has no information about variable binding in R. A possible fix is adding a function `inplace` to R, which ensure the returned variable has named filed = 0 so is safe to modify inplace. Then, we have to call the function as `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It shows clearly that we are breaking the pass-by-value rule in R. ```cpp #include Rcpp.h using namespace Rcpp; // [[Rcpp::export]] void stl_sort_inplace(NumericVector x) { std::sort(x.begin(), x.end()); } ``` ```r a - seq(1, 0.1, -0.1) b - a # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 stl_sort_inplace(a) a # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 b # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 a - seq(1, 0.1, -0.1) pure_function - function (x) { y - x stl_sort_inplace(y) print(y) } pure_function(a) a # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 This is common among C++ software called by R that modifies R objects in place. For example, below DT2 is modified: library(data.table) ...junk... DT - data.table(a = 1:3) DT2 - DT DT[, b:=a] DT2 a b 1: 1 1 2: 2 2 3: 3 3 ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] inplace modification more affect other varibles
Thanks a lot! I thought that was a bug of data.table, when I tried to learn data.table. Obviously, I was wrong. It's a feature of data.table, in which all set functions change their input by reference. It also provide function copy when a copy is needed. Based on suggestion from Romain, I may just stay on the safe side and do not modify argument passed to C++ from R. The users of data.table should be aware of that data.table object is passed by reference, and call function copy when needed. For other R objects, it seems cause too much trouble. It's hard to detect variables pointing to the same place and I don't want to provide a copy function. On Wed, Oct 22, 2014 at 11:40 AM, Gabor Grothendieck ggrothendi...@gmail.com wrote: On Tue, Oct 21, 2014 at 9:22 PM, Chenliang Xu luckyr...@gmail.com wrote: Hello, With the following inplace sorting example, I understand the value of `a` is sorted inplace, but it's strange to see the value of `b` is also modified. This can cause some hard to detect bug, since the cpp function may modify a variable defined in other scope. It seems that rcpp doesn't respect the named field, which is adopted by R to implement copy-on-modify. I don's see an easy fix on C++ side, since the called cpp function has no information about variable binding in R. A possible fix is adding a function `inplace` to R, which ensure the returned variable has named filed = 0 so is safe to modify inplace. Then, we have to call the function as `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It shows clearly that we are breaking the pass-by-value rule in R. ```cpp #include Rcpp.h using namespace Rcpp; // [[Rcpp::export]] void stl_sort_inplace(NumericVector x) { std::sort(x.begin(), x.end()); } ``` ```r a - seq(1, 0.1, -0.1) b - a # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 stl_sort_inplace(a) a # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 b # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 a - seq(1, 0.1, -0.1) pure_function - function (x) { y - x stl_sort_inplace(y) print(y) } pure_function(a) a # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 This is common among C++ software called by R that modifies R objects in place. For example, below DT2 is modified: library(data.table) ...junk... DT - data.table(a = 1:3) DT2 - DT DT[, b:=a] DT2 a b 1: 1 1 2: 2 2 3: 3 3 ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
[Rcpp-devel] inplace modification more affect other varibles
Hello, With the following inplace sorting example, I understand the value of `a` is sorted inplace, but it's strange to see the value of `b` is also modified. This can cause some hard to detect bug, since the cpp function may modify a variable defined in other scope. It seems that rcpp doesn't respect the named field, which is adopted by R to implement copy-on-modify. I don's see an easy fix on C++ side, since the called cpp function has no information about variable binding in R. A possible fix is adding a function `inplace` to R, which ensure the returned variable has named filed = 0 so is safe to modify inplace. Then, we have to call the function as `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It shows clearly that we are breaking the pass-by-value rule in R. ```cpp #include Rcpp.h using namespace Rcpp; // [[Rcpp::export]] void stl_sort_inplace(NumericVector x) { std::sort(x.begin(), x.end()); } ``` ```r a - seq(1, 0.1, -0.1) b - a # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 stl_sort_inplace(a) a # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 b # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 a - seq(1, 0.1, -0.1) pure_function - function (x) { y - x stl_sort_inplace(y) print(y) } pure_function(a) a # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 ``` ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] inplace modification more affect other varibles
On 21 October 2014 at 20:22, Chenliang Xu wrote: | Hello, | | With the following inplace sorting example, I understand the value of `a` is | sorted inplace, but it's strange to see the value of `b` is also modified. This | can cause some hard to detect bug, since the cpp function may modify a variable | defined in other scope. Very well known issue -- maybe do a search for 'Rcpp::clone' ... In a nutshell, SEXP objects are passed by a __pointer__ and changes do therefore persist. If you want distinct copies, use Rcpp::clone(). Dirk | It seems that rcpp doesn't respect the named field, which is adopted by R to | implement copy-on-modify. I don's see an easy fix on C++ side, since the called | cpp function has no information about variable binding in R. A possible fix is | adding a function `inplace` to R, which ensure the returned variable has named | filed = 0 so is safe to modify inplace. Then, we have to call the function as | `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It | shows clearly that we are breaking the pass-by-value rule in R. | | ```cpp | #include Rcpp.h | using namespace Rcpp; | | // [[Rcpp::export]] | void stl_sort_inplace(NumericVector x) { | std::sort(x.begin(), x.end()); | } | | ``` | | ```r | a - seq(1, 0.1, -0.1) | b - a | # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 | | stl_sort_inplace(a) | | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | b | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | a - seq(1, 0.1, -0.1) | pure_function - function (x) { | y - x | stl_sort_inplace(y) | print(y) | } | pure_function(a) | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | ``` | | ___ | Rcpp-devel mailing list | Rcpp-devel@lists.r-forge.r-project.org | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
Re: [Rcpp-devel] inplace modification more affect other varibles
Hi Dirk, Thanks for your quick answer. I don't think Rcpp::clone is what I was looking for. I know `stl_sort_inplace(a)` modify the value of `a`, but it surprise me to see it modify `b`. And it may modify some other variables c, d, e, f..., and it's hard to know which variables point to the same place. On Tue, Oct 21, 2014 at 8:31 PM, Dirk Eddelbuettel e...@debian.org wrote: On 21 October 2014 at 20:22, Chenliang Xu wrote: | Hello, | | With the following inplace sorting example, I understand the value of `a` is | sorted inplace, but it's strange to see the value of `b` is also modified. This | can cause some hard to detect bug, since the cpp function may modify a variable | defined in other scope. Very well known issue -- maybe do a search for 'Rcpp::clone' ... In a nutshell, SEXP objects are passed by a __pointer__ and changes do therefore persist. If you want distinct copies, use Rcpp::clone(). Dirk | It seems that rcpp doesn't respect the named field, which is adopted by R to | implement copy-on-modify. I don's see an easy fix on C++ side, since the called | cpp function has no information about variable binding in R. A possible fix is | adding a function `inplace` to R, which ensure the returned variable has named | filed = 0 so is safe to modify inplace. Then, we have to call the function as | `stl_sort_inplace(inplace(a))`, which seems odd but is also informative. It | shows clearly that we are breaking the pass-by-value rule in R. | | ```cpp | #include Rcpp.h | using namespace Rcpp; | | // [[Rcpp::export]] | void stl_sort_inplace(NumericVector x) { | std::sort(x.begin(), x.end()); | } | | ``` | | ```r | a - seq(1, 0.1, -0.1) | b - a | # [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 | | stl_sort_inplace(a) | | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | b | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | a - seq(1, 0.1, -0.1) | pure_function - function (x) { | y - x | stl_sort_inplace(y) | print(y) | } | pure_function(a) | a | # [1] 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 | | ``` | | ___ | Rcpp-devel mailing list | Rcpp-devel@lists.r-forge.r-project.org | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org ___ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel