Hi Henrik:
Thank you for the prompt reply. You are right- the .CEL files are all read
only (see their permissions, below)
rtraborn@Mason:
/N/dc2/scratch/rtraborn/ML_Project/NIK_2014/rawData/NIK_2014/HuEx-1_0-st-v2>
ls -lkhtra
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709338_HSB194-VFC-R.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709337_HSB194-VFC-L.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709336_HSB194-V1C-R.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709335_HSB194-V1C-L.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709334_HSB194-STR-L.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709333_HSB194-STC-R.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709332_HSB194-STC-L.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709331_HSB194-S1C-R.CEL
-r--r--r-- 1 ... 63M Jan 25 2011 GSM709330_HSB194-S1C-L.CEL
After changing the permissions, the job performed without issue (I'm
showing you only the results of 1/9 for the sake of brevity):
> csBC <- process(bc,verbose=verbose)
Background correcting data set...
Number of arrays: 9
Array #1 ('GSM709330_HSB194-S1C-L') of 9...
Adjusting PM signals only
Obtaining signals...
Obtaining signals...done
Applying normal+exponential signal model...
Applying normal+exponential signal model...done
Writing adjusted probe signals...
Adding temporary suffix from file...
Pathname:
probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
Suffix: .tmp
Rename existing file?: FALSE
Temporary pathname:
probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
Adding temporary suffix from file...done
Creating CEL file for results, if missing...
Creating CEL file for results, if missing...done
Writing adjusted intensities...
Writing adjusted intensities...done
Dropping temporary suffix from file...
Temporary pathname:
probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
Suffix: .tmp
Regular expression for suffix: \.tmp$
Pathname:
probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
Renaming existing file...
Result: TRUE
Renaming existing file...done
Dropping temporary suffix from file...done
Writing adjusted probe signals...done
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 709078 37.9 1166886 62.4 1166886 62.4
Vcells 31586985 241.0 47054688 359.0 46991036 358.6
AffymetrixCelFile:
Name: GSM709330_HSB194-S1C-L
Tags:
Full name: GSM709330_HSB194-S1C-L
Pathname:
probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
File size: 62.73 MB (65775403 bytes)
RAM: 0.00 MB
File format: v4 (binary; XDA)
Platform: Affymetrix
Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
Timestamp: 2011-01-25 12:15:03
Array #1 ('GSM709330_HSB194-S1C-L') of 9...done
-----
There must be a file permissions transfer behavior that takes place on a
multi-user cluster environment that doesn't take place on other types of
machines, because I was unable to reproduce this error elsewhere. I didn't
bother changing the raw .CEL files' permissions because I never experienced
a problem of this nature with .CEL files.
Let me know if you patch this up so I can pull the latest code from the
aroma repo.
Thanks for your help with this and for taking a look at the issue so
quickly.
Best regards,
Taylor
On Thursday, September 4, 2014 1:37:36 PM UTC-4, Henrik Bengtsson wrote:
>
> Hi,
>
> interesting. A quick guess is that the input file, i.e.
>
> rawData/NIK_2014/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>
> is write protected (e.g. file owned by someone else). This could be
> the reason, because internally that file is copied and used as a
> template and the copy may inherit the file permissions. What does ls
> -l say about the above file?
>
> If this is the case, I need to update the copying such that the new
> file has proper permissions.
>
> Thanks for the report
>
> Henrik
>
>
> On Thu, Sep 4, 2014 at 10:15 AM, Taylor Raborn <taylor...@gmail.com
> <javascript:>> wrote:
> > Hi Henrik:
> >
> > I just come across an unusual error when using aroma.affymetrix on our
> > high-memory HPC machine. It may or may not be platform-specific, and
> I'll
> > give you all the information I can in the hopes of isolating and
> identifying
> > the error.
> >
> > Traceback:
> >
> >> traceback()
> >
> > 18: stop(cond)
> >
> > 17: throw.Exception(Exception(...))
> >
> > 16: throw(Exception(...))
> >
> > 15: throw.default("No permission to modify existing file: ", pathname)
> >
> > 14: throw("No permission to modify existing file: ", pathname)
> >
> > 13: getWritablePathname.Arguments(static, ...)
> >
> > 12: getWritablePathname(static, ...) at <text>#1
> >
> > 11: Arguments$getWritablePathname(pathname, mustExist = TRUE)
> >
> > 10: renameFile.default(srcPathname, pathname, ...)
> >
> > 9: renameFile(srcPathname, pathname, ...)
> >
> > 8: renameTo.GenericDataFile(res, filename = pathname, verbose =
> > less(verbose))
> >
> > 7: renameTo(res, filename = pathname, verbose = less(verbose))
> >
> > 6: createFrom.AffymetrixCelFile(this, filename = pathnameT, path = NULL,
> >
> > verbose = less(verbose))
> >
> > 5: createFrom(this, filename = pathnameT, path = NULL, verbose =
> > less(verbose))
> >
> > 4: bgAdjustRma.AffymetrixCelFile(df, path = outputPath, pmonly = pmonly,
> >
> > addJitter = addJitter, jitterSd = jitterSd, overwrite = force,
> >
> > verbose = verbose, .deprecated = FALSE)
> >
> > 3: bgAdjustRma(df, path = outputPath, pmonly = pmonly, addJitter =
> > addJitter,
> >
> > jitterSd = jitterSd, overwrite = force, verbose = verbose,
> >
> > .deprecated = FALSE)
> >
> > 2: process.RmaBackgroundCorrection(bc, verbose = verbose)
> >
> > 1: process(bc, verbose = verbose)
> >
> >
> > ----
> >
> > sessionInfo:
> >
> >> sessionInfo()
> >
> > R version 3.0.1 (2013-05-16)
> >
> > Platform: x86_64-unknown-linux-gnu (64-bit)
> >
> >
> > locale:
> >
> > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> >
> > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
> >
> > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
> >
> > [7] LC_PAPER=C LC_NAME=C
> >
> > [9] LC_ADDRESS=C LC_TELEPHONE=C
> >
> > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> >
> >
> > attached base packages:
> >
> > [1] tools parallel stats graphics grDevices utils datasets
> >
> > [8] methods base
> >
> >
> > other attached packages:
> >
> > [1] R.cache_0.10.0 base64enc_0.1-2 aroma.apd_0.5.0
> >
> > [4] preprocessCore_1.24.0 affyio_1.30.0 Biobase_2.20.1
> >
> > [7] BiocGenerics_0.6.0 aroma.light_1.32.0 matrixStats_0.10.0
> >
> > [10] aroma.affymetrix_2.12.0 aroma.core_2.12.1 R.devices_2.11.0
> >
> > [13] R.filesets_2.6.0 R.utils_1.33.0 R.oo_1.18.0
> >
> > [16] affxparser_1.34.2 affy_1.40.0 R.methodsS3_1.6.1
> >
> > [19] PSCBS_0.43.0 DNAcopy_1.36.0
> BiocInstaller_1.12.1
> >
> >
> > loaded via a namespace (and not attached):
> >
> > [1] digest_0.6.4 R.huge_0.8.0 R.rsp_0.19.0 zlibbioc_1.11.1
> >
> >
> >
> > Here's how I produced the error. Note that my directory structure
> matches
> > that on the aroma-project.org site, shown here:
> >
> >> chipType <- "HuEx-1_0-st-v2"
> >
> >
> >> cdf <- AffymetrixCdfFile$byChipType(chipType,
> tags="coreR3,A20071112,EP")
> >
> >
> >> print(cdf)
> >
> > AffymetrixCdfFile:
> >
> > Path: annotationData/chipTypes/HuEx-1_0-st-v2
> >
> > Filename: HuEx-1_0-st-v2,coreR3,A20071112,EP.cdf
> >
> > File size: 38.25 MB (40108891 bytes)
> >
> > Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
> >
> > RAM: 0.00MB
> >
> > File format: v4 (binary; XDA)
> >
> > Dimension: 2560x2560
> >
> > Number of cells: 6553600
> >
> > Number of units: 18708
> >
> > Cells per unit: 350.31
> >
> > Number of QC units: 1
> >
> >
> >> cs <- AffymetrixCelSet$byName("NIK_2014", cdf=cdf)
> >
> >
> >> print(cs)
> >
> > AffymetrixCelSet:
> >
> > Name: NIK_2014
> >
> > Tags:
> >
> > Path: rawData/NIK_2014/HuEx-1_0-st-v2
> >
> > Platform: Affymetrix
> >
> > Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
> >
> > Number of arrays: 9
> >
> > Names: GSM709330_HSB194-S1C-L, GSM709331_HSB194-S1C-R,
> > GSM709332_HSB194-STC-L, ..., GSM709338_HSB194-VFC-R [9]
> >
> > Time period: 2011-01-25 12:15:03 -- 2011-01-25 13:48:02
> >
> > Total file size: 564.87MB
> >
> > RAM: 0.02MB
> >
> >
> >> csBC <- process(bc,verbose=verbose)
> >
> > Background correcting data set...
> >
> > Number of arrays: 9
> >
> > Array #1 ('GSM709330_HSB194-S1C-L') of 9...
> >
> > Adjusting PM signals only
> >
> > Obtaining signals...
> >
> > Obtaining signals...done
> >
> > Applying normal+exponential signal model...
> >
> > Applying normal+exponential signal model...done
> >
> > Writing adjusted probe signals...
> >
> > Adding temporary suffix from file...
> >
> > Pathname:
> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
> >
> > Suffix: .tmp
> >
> > Rename existing file?: FALSE
> >
> > Temporary pathname:
> >
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
> >
> > Adding temporary suffix from file...done
> >
> > Creating CEL file for results, if missing...
> >
> > [2014-09-04 12:38:54] Exception: No permission to modify existing file:
> >
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp.tmp
>
>
> >
> >
> > at #13. getWritablePathname.Arguments(static, ...)
> >
> > - getWritablePathname.Arguments() is in environment 'R.utils'
> >
> >
> > at #12. getWritablePathname(static, ...)
> >
> > - getWritablePathname() is in environment 'R.utils'
> >
> > - originating from '<text>'
> >
> >
> > at #11. Arguments$getWritablePathname(pathname, mustExist = TRUE)
> >
> > - Arguments$getWritablePathname() is local of the calling
> function
> >
> >
> > at #10. renameFile.default(srcPathname, pathname, ...)
> >
> > - renameFile.default() is in environment 'R.utils'
> >
> >
> > at #09. renameFile(srcPathname, pathname, ...)
> >
> > - renameFile() is in environment 'R.utils'
> >
> >
> > at #08. renameTo.GenericDataFile(res, filename = pathname, verbose =
> > less(verbose))
> >
> > - renameTo.GenericDataFile() is in environment 'R.filesets'
> >
> >
> > at #07. renameTo(res, filename = pathname, verbose = less(verbose))
> >
> > - renameTo() is in environment 'R.filesets'
> >
> >
> > at #06. createFrom.AffymetrixCelFile(this, filename = pathnameT, path
> =
> > NULL,
> >
> > verbose = less(verbose))
> >
> > - createFrom.AffymetrixCelFile() is in environment
> > 'aroma.affymetrix'
> >
> >
> > at #05. createFrom(this, filename = pathnameT, path = NULL, verbose =
> > less(verbose))
> >
> > - createFrom() is in environment 'aroma.affymetrix'
> >
> >
> > at #04. bgAdjustRma.AffymetrixCelFile(df, path = outputPath, pmonly =
> > pmonly,
> >
> > addJitter = addJitter, jitterSd = jitterSd, overwrite =
> force,
> >
> > verbose = verbose, .deprecated = FALSE)
> >
> > - bgAdjustRma.AffymetrixCelFile() is in environment
> > 'aroma.affymetrix'
> >
> >
> > at #03. bgAdjustRma(df, path = outputPath, pmonly = pmonly, addJitter
> =
> > addJitter,
> >
> > jitterSd = jitterSd, overwrite = force, verbose = verbose,
> >
> > .deprecated = FALSE)
> >
> > - bgAdjustRma() is in environment 'aroma.affymetrix'
> >
> >
> > at #02. process.RmaBackgroundCorrection(bc, verbose = verbose)
> >
> > - process.RmaBackgroundCorrection() is in environment
> > 'aroma.affymetrix'
> >
> >
> > at #01. process(bc, verbose = verbose)
> >
> > - process() is in environment 'aroma.core'
> >
> >
> > Error: No permission to modify existing file:
> >
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp.tmp
>
>
> >
> > Creating CEL file for results, if missing...done
> >
> > Writing adjusted probe signals...done
> >
> > Array #1 ('GSM709330_HSB194-S1C-L') of 9...done
> >
> > Background correcting data set...done
> >
> > --------------------
> >
> >
> > I tried to find a solution from the group archives and other places to
> no
> > avail. It's worth mentioning that I get this problem even if I open up
> > permissions to the entire directory. The *CEL.tmp.tmp file appears to be
> > written with write-protections, which is baffling to me. The machine is
> a
> > large, high-memory cluster from my institution, and the error may be
> > associated with this.
> >
> >
> > Thanks in advance for any insight you can provide with this.
> >
> >
> > Best regards,
> >
> >
> > Taylor
> >
> >
> > --
> > --
> > When reporting problems on aroma.affymetrix, make sure 1) to run the
> latest
> > version of the package, 2) to report the output of sessionInfo() and
> > traceback(), and 3) to post a complete code example.
> >
> >
> > You received this message because you are subscribed to the Google
> Groups
> > "aroma.affymetrix" group with website http://www.aroma-project.org/.
> > To post to this group, send email to aroma-af...@googlegroups.com
> <javascript:>
> > To unsubscribe and other options, go to
> http://www.aroma-project.org/forum/
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups
> > "aroma.affymetrix" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to aroma-affymetr...@googlegroups.com <javascript:>.
> > For more options, visit https://groups.google.com/d/optout.
>
--
--
When reporting problems on aroma.affymetrix, make sure 1) to run the latest
version of the package, 2) to report the output of sessionInfo() and
traceback(), and 3) to post a complete code example.
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group with website http://www.aroma-project.org/.
To post to this group, send email to aroma-affymetrix@googlegroups.com
To unsubscribe and other options, go to http://www.aroma-project.org/forum/
---
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to aroma-affymetrix+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
