Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output

[Daniel Drake]

On Sun 19 Feb 2012 12:46:15 PM PST, Eric Schulte wrote:


A further followup, at the risk of descending into minutia.  The
culprit seems to be the emacs function string-to-number.

On my 32-bit Arch machine:
(string-to-number "123456789"): 123456789 (#o726746425, #x75bcd15)
(string-to-number "987654321"): 987654321.0
GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of
2012-02-01 on shirley.hoetzel.info

On a 64-bit Ubuntu (11.10) machine:
(string-to-number "123456789"): 123456789 (#o726746425, #x75bcd15)
(string-to-number "987654321"): 987654321 (#o7267464261, #x3ade68b1)
GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.5) of
2011-08-14 on allspice, modified by Debian



Ah, nice debugging!

That must be the issue, as I'm working on a 64-bit machine.

Maybe one workaround could be to prefix the numbers with a letter in
R-land so that emacs interprets them as strings.

Cheers,



Poking around the web I found that emacs guarantees 28 bits as a 
minimum precision.  For a while I did as you suggested, prefixing all 
subjects ids with 'id'.  But it's cleaner if I reinstall a 64-bit 
version of Arch.  From your experiments, I already know its 64-bit 
version of emacs supports at least 30 bits.


Thanks again for your help.
Dan

Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output

[Daniel Drake]

On 02/19/2012 10:28 AM, Daniel Drake wrote:

On 02/19/2012 09:40 AM, Daniel Drake wrote:

On 02/19/2012 08:48 AM, Eric Schulte wrote:

Just to be safe, I nuked my org-mode directory and re-cloned from the
git repository: I'm now at release_7.8.03.386.g2239d (I was at
release_7.8.03-351-g47eb3 previously). Is there a more recent
version? I also removed the org directory that came with the packaged
version of emacs, just in case.



This sounds just like my setup. I get the following from `org-version'
which should be equivalent.

Org-mode version 7.8.03 (release_7.8.03.419.gdeb6e)



I start emacs with the following options:
$ emacs -Q -eval '(setq load-path (cons
"~/share/emacs/site-lisp/org-mode/lisp" load-path))' -eval "(require
ob-R)" test.org



I just did the same using the following command line to launch Emacs,
and I still get no decimal place in my results.

$ emacs -Q -eval '(setq laod-path (cons "~/.emacs.d/src/org/lisp/"
load-path))' -eval "(require 'ob-R)" /tmp/dan.org



I get the same behavior as before. I was a little puzzled when I saw
this happen, as I've been working on this study for over a year in org
and it's the first time I've seen this behavior. I went back and
tried previous versions (e.g., 7.4), and now I see the same thing.

I wonder if Arch (a rolling release linux distribution that tries to
be close to the leading edge) has updated an underlying library that
impacts the babel code in some odd way. (Granted, I don't see how
this could be, as the emacs environment seems pretty well insulated
from the underlying OS; PEBKAC just as likely.)



I doubt this is the case. I also use Arch (just ran pacman -Syu this
morning) so we likely have the largely same underlying versions of
libraries installed. Maybe the issue could lay somewhere in your R
config?

Sorry I can't be of more help.

Best,



At first I thought it was R as well, but the fact that there is no
decimal point in the output file plus the fact that (outside of emacs) I
can use read.table to pull in the table and the result has no decimal
formatting makes me think otherwise. That you're running the same setup
as me, though, gives me hope. I can backup, re-install Arch, and see if
the problem goes away.

Thanks for your help. I'm not certain when I'll have time to try this,
but I'll follow up on this thread when I do.

Best,
Dan


I'm following up on my last post, just to have it in the record: I've
boiled down the behavior to these two examples: output the results by
value vs output. R seems not to have anything to do with it. Somehow the
"by value" code is intervening. I'll try to debug.

command line:
$ emacs -Q -eval '(setq load-path (cons
"~/share/emacs/site-lisp/org-mode/lisp" load-path))' -eval "(require
'ob-sh)" test.org

In the buffer:
*** by output
#+name: by-output
#+begin_src sh :results output
echo 987654321
#+end_src

#+RESULTS: by-output
: 987654321

*** by value
#+name: by-value
#+begin_src sh :results value
echo 987654321
#+end_src

#+RESULTS: by-value
: 987654321.0





A further followup, at the risk of descending into minutia.  The culprit 
seems to be the emacs function string-to-number.


On my 32-bit Arch machine:
(string-to-number "123456789"): 123456789 (#o726746425, #x75bcd15)
(string-to-number "987654321"): 987654321.0
GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 
on shirley.hoetzel.info


On a 64-bit Ubuntu (11.10) machine:
(string-to-number "123456789"): 123456789 (#o726746425, #x75bcd15)
(string-to-number "987654321"): 987654321 (#o7267464261, #x3ade68b1)
GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.5) of 
2011-08-14 on allspice, modified by Debian


The extra 3 bits required to represent the larger number apparently push 
my version of emacs into floating point 
territory.(log2(987654321)=29.87943; log2(123456789)=26.87943)


Don't know if the difference is between emacs versions or 32- vs. 64-bit 
implementations, but I can probably figure it out from here.  In any 
event, this doesn't seem to be an org-mode problem.


Thanks again for all your help and for org-mode/babel in general!
Dan





In any event, since you can't reproduce the behavior (thanks again for
trying), I don't expect it's anything you can fix. Maybe it will show
up in other distributions as they update to more current versions. In
the mean time, I'll try to improve my elisp skills and figure out
where the extraneous formatting occurs.

Best,
Dan

- GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of
2012-02-01 on shirley.hoetzel.info
- Org-mode version 7.8.03 (release_7.8.03.386.g2239d)

Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output

[Daniel Drake]

On 02/19/2012 09:40 AM, Daniel Drake wrote:

On 02/19/2012 08:48 AM, Eric Schulte wrote:

Just to be safe, I nuked my org-mode directory and re-cloned from the
git repository: I'm now at release_7.8.03.386.g2239d (I was at
release_7.8.03-351-g47eb3 previously). Is there a more recent
version? I also removed the org directory that came with the packaged
version of emacs, just in case.



This sounds just like my setup. I get the following from `org-version'
which should be equivalent.

Org-mode version 7.8.03 (release_7.8.03.419.gdeb6e)



I start emacs with the following options:
$ emacs -Q -eval '(setq load-path (cons
"~/share/emacs/site-lisp/org-mode/lisp" load-path))' -eval "(require
ob-R)" test.org



I just did the same using the following command line to launch Emacs,
and I still get no decimal place in my results.

$ emacs -Q -eval '(setq laod-path (cons "~/.emacs.d/src/org/lisp/"
load-path))' -eval "(require 'ob-R)" /tmp/dan.org



I get the same behavior as before. I was a little puzzled when I saw
this happen, as I've been working on this study for over a year in org
and it's the first time I've seen this behavior. I went back and
tried previous versions (e.g., 7.4), and now I see the same thing.

I wonder if Arch (a rolling release linux distribution that tries to
be close to the leading edge) has updated an underlying library that
impacts the babel code in some odd way. (Granted, I don't see how
this could be, as the emacs environment seems pretty well insulated
from the underlying OS; PEBKAC just as likely.)



I doubt this is the case. I also use Arch (just ran pacman -Syu this
morning) so we likely have the largely same underlying versions of
libraries installed. Maybe the issue could lay somewhere in your R
config?

Sorry I can't be of more help.

Best,



At first I thought it was R as well, but the fact that there is no
decimal point in the output file plus the fact that (outside of emacs) I
can use read.table to pull in the table and the result has no decimal
formatting makes me think otherwise. That you're running the same setup
as me, though, gives me hope. I can backup, re-install Arch, and see if
the problem goes away.

Thanks for your help. I'm not certain when I'll have time to try this,
but I'll follow up on this thread when I do.

Best,
Dan


I'm following up on my last post, just to have it in the record: I've 
boiled down the behavior to these two examples: output the results by 
value vs output.  R seems not to have anything to do with it.  Somehow 
the "by value" code is intervening.  I'll try to debug.


command line:
$ emacs -Q -eval '(setq load-path (cons 
"~/share/emacs/site-lisp/org-mode/lisp" load-path))' -eval "(require 
'ob-sh)" test.org


In the buffer:
*** by output
#+name: by-output
#+begin_src sh  :results output
  echo 987654321
#+end_src

#+RESULTS: by-output
: 987654321

*** by value
#+name: by-value
#+begin_src sh  :results value
  echo 987654321
#+end_src

#+RESULTS: by-value
: 987654321.0







In any event, since you can't reproduce the behavior (thanks again for
trying), I don't expect it's anything you can fix. Maybe it will show
up in other distributions as they update to more current versions. In
the mean time, I'll try to improve my elisp skills and figure out
where the extraneous formatting occurs.

Best,
Dan

- GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of
2012-02-01 on shirley.hoetzel.info
- Org-mode version 7.8.03 (release_7.8.03.386.g2239d)

Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output

[Daniel Drake]

On 02/19/2012 08:48 AM, Eric Schulte wrote:

Just to be safe, I nuked my org-mode directory and re-cloned from the
git repository: I'm now at release_7.8.03.386.g2239d (I was at
release_7.8.03-351-g47eb3 previously).  Is there a more recent
version? I also removed the org directory that came with the packaged
version of emacs, just in case.



This sounds just like my setup.  I get the following from `org-version'
which should be equivalent.

Org-mode version 7.8.03 (release_7.8.03.419.gdeb6e)



I start emacs with the following options:
$ emacs -Q -eval '(setq load-path (cons
"~/share/emacs/site-lisp/org-mode/lisp" load-path))' -eval "(require
ob-R)" test.org



I just did the same using the following command line to launch Emacs,
and I still get no decimal place in my results.

$ emacs -Q -eval '(setq laod-path (cons "~/.emacs.d/src/org/lisp/" load-path))' -eval 
"(require 'ob-R)" /tmp/dan.org



I get the same behavior as before.  I was a little puzzled when I saw
this happen, as I've been working on this study for over a year in org
and it's the first time I've seen this behavior.  I went back and
tried previous versions (e.g., 7.4), and now I see the same thing.

I wonder if Arch (a rolling release linux distribution that tries to
be close to the leading edge) has updated an underlying library that
impacts the babel code in some odd way.  (Granted, I don't see how
this could be, as the emacs environment seems pretty well insulated
from the underlying OS; PEBKAC just as likely.)



I doubt this is the case.  I also use Arch (just ran pacman -Syu this
morning) so we likely have the largely same underlying versions of
libraries installed.  Maybe the issue could lay somewhere in your R
config?

Sorry I can't be of more help.

Best,



At first I thought it was R as well, but the fact that there is no 
decimal point in the output file plus the fact that (outside of emacs) I 
can use read.table to pull in the table and the result has no decimal
formatting makes me think otherwise.  That you're running the same setup 
as me, though, gives me hope.  I can backup, re-install Arch, and see if 
the problem goes away.


Thanks for your help.  I'm not certain when I'll have time to try this, 
but I'll follow up on this thread when I do.


Best,
Dan



In any event, since you can't reproduce the behavior (thanks again for
trying), I don't expect it's anything you can fix.  Maybe it will show
up in other distributions as they update to more current versions.  In
the mean time, I'll try to improve my elisp skills and figure out
where the extraneous formatting occurs.

Best,
Dan

- GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of
2012-02-01 on shirley.hoetzel.info
- Org-mode version 7.8.03 (release_7.8.03.386.g2239d)

Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output

[Daniel Drake]

On 02/18/2012 08:23 AM, Eric Schulte wrote:

Daniel Drake  writes:


Hi All,

I'm using R in org-mode/babel to analyze data from a psychological
study.  The subjects in this study are identified by nine digit integers
(e.g., 987654321) that I treat as strings (or factors) in my R data
frames.

Tables output by an R code block that contain these subject IDs are
not formatted properly: the subject IDs seem to be treated as numbers
and a decimal point and a trailing zero are appended.  For example,
what should be
|   subj.id |
|---|
| 987654321 |
becomes
| subj.id |
| 987654321.0 |
(I've included real, self-contained code below.)

When I write the data frames directly to a file from within the R code
block (using a write.table function call that mimics the one in
ob-R.el), the integer IDs are preserved; but when code from ob-R.el
writes them out, the values get reformated as floats.

I've noticed that eight digit integers are not modified in the same
way, which makes me wonder if there is a 'digits' threshold I could
modify to prevent this from happening.

I've pretty much ruled out the possibility that this transformation
occurs in R; however I'm not proficient enough in elisp to follow the
operations that happen after the data frame is written to a file in
the org-babel-R-write-object-command.

Any pointers to help me figure this out would be very appreciated!
(I've seen this thread:
http://comments.gmane.org/gmane.emacs.orgmode/31373, but do not know
if the conversion to calc has been made already or if it is the source
of the problem.)



Hi Dan,

When I launch Emacs without any personal configuration (emacs -Q),
evaluate ob-R.el to add R support, and then execute your code blocks I
get the following behavior.





I would recommend installing the latest version of Org-mode from source
as that tends to fix many problems, and will at least ensure that we are
both working from the same code base moving forward.  See
http://orgmode.org/worg/org-faq.html#Keeping-current

Best,



Hi Eric,
First of all, thanks for your reply.

Just to be safe, I nuked my org-mode directory and re-cloned from the 
git repository: I'm now at release_7.8.03.386.g2239d (I was at 
release_7.8.03-351-g47eb3 previously).  Is there a more recent version? 
 I also removed the org directory that came with the packaged version 
of emacs, just in case.


I start emacs with the following options:
$ emacs -Q -eval '(setq load-path (cons 
"~/share/emacs/site-lisp/org-mode/lisp" load-path))' -eval "(require 
'ob-R)" test.org


I get the same behavior as before.  I was a little puzzled when I saw 
this happen, as I've been working on this study for over a year in org 
and it's the first time I've seen this behavior.  I went back and tried 
previous versions (e.g., 7.4), and now I see the same thing.


I wonder if Arch (a rolling release linux distribution that tries to be 
close to the leading edge) has updated an underlying library that 
impacts the babel code in some odd way.  (Granted, I don't see how this 
could be, as the emacs environment seems pretty well insulated from the 
underlying OS; PEBKAC just as likely.)


In any event, since you can't reproduce the behavior (thanks again for 
trying), I don't expect it's anything you can fix.  Maybe it will show 
up in other distributions as they update to more current versions.  In 
the mean time, I'll try to improve my elisp skills and figure out where 
the extraneous formatting occurs.


Best,
Dan

- GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 
2012-02-01 on shirley.hoetzel.info

- Org-mode version 7.8.03 (release_7.8.03.386.g2239d)

[O] [0][babel][R] Undesired conversion of integers to floats in R code block output

[Daniel Drake]

Hi All,

I'm using R in org-mode/babel to analyze data from a psychological
study.  The subjects in this study are identified by nine digit integers
(e.g., 987654321) that I treat as strings (or factors) in my R data
frames.

Tables output by an R code block that contain these subject IDs are
not formatted properly: the subject IDs seem to be treated as numbers
and a decimal point and a trailing zero are appended.  For example,
what should be
|   subj.id |
|---|
| 987654321 |
becomes
| subj.id |
| 987654321.0 |
(I've included real, self-contained code below.)

When I write the data frames directly to a file from within the R code
block (using a write.table function call that mimics the one in
ob-R.el), the integer IDs are preserved; but when code from ob-R.el
writes them out, the values get reformated as floats.

I've noticed that eight digit integers are not modified in the same
way, which makes me wonder if there is a 'digits' threshold I could
modify to prevent this from happening.

I've pretty much ruled out the possibility that this transformation
occurs in R; however I'm not proficient enough in elisp to follow the
operations that happen after the data frame is written to a file in
the org-babel-R-write-object-command.

Any pointers to help me figure this out would be very appreciated!
(I've seen this thread:
http://comments.gmane.org/gmane.emacs.orgmode/31373, but do not know
if the conversion to calc has been made already or if it is the source
of the problem.)

Thanks,
Dan

I'm running:
- Arch Linux (32 bit)
- GNU Emacs 23.4.1
- Org-mode version release_7.8.03-351-g47eb3
- R version 2.14.1 (2011-12-22)

* Test
** table as generated by org-mode/babel
#+name: make
#+begin_src R  :results value  :colnames yes
  temp <- data.frame('A'=c('987654321'),'B'=c('98765432'))
  ## -- this call mimics the one in ob-R.el
write.table(temp,file='test_r.tsv',sep='\t',na='nil',row.names=FALSE,col.names=TRUE,quote=FALSE)
  temp
#+end_src

#+RESULTS: make
|   A |B |
|-+--|
| 987654321.0 | 98765432 |

** table as generated by R directly
#+name: read
#+begin_src sh  :results output
  cat test_r.tsv
#+end_src

#+RESULTS: read
: A B
: 987654321 98765432