Actually, it turns out unit tests explicitly do setBroadcast(false)... here's probably why. But i never remember having this problem or being motivated to set it false by anything... Hm..
On Thu, Sep 6, 2012 at 11:37 AM, Dmitriy Lyubimov <dlie...@gmail.com> wrote: > I don't believe it doesn't work in local mode because its unit tests > are run in local mode. With exception of # of reducers, everything > else works there just the same. > > That said, you can disable DistributedCache in some cases using > SSVDSolver#setBroadcast(false). (in spite of what javadoc says, it is > enabled by default... my bad). > > On Thu, Sep 6, 2012 at 11:18 AM, Pat Ferrel <p...@occamsmachete.com> wrote: >> To reiterate the situation. In local mode using the local file system SSVD >> dies with a file not found. In pseudo-cluster mode using hdfs SSVD on the >> same data it runs correctly. All the rest of the analysis pipeline works >> fine in either mode. I am using local mode to debug my surrounding code. >> >> From the error output it looks like the code is using the DistributedCache >> of hadoop. This is said to not work with local hadoop, though the comment >> was for a pre 0.20.205 version (my version). >> >> The implication being that when MAHOUT_LOCAL is set you shouldn't use the >> DistributedCache. >> >> Could this be the problem? stochasticSVD uses DistributedCache in several >> spots >> >> ========================== >> >> The /tmp file does not exist, the local >> >> java.io.FileNotFoundException: File >> /tmp/hadoop-pat/mapred/local/archive/5543644668644532045_1587570556_2120541978/file/Users/pat/Projects/big-data/b/ssvd/Q-job/R-m-00000 >> does not exist. >> >> Maclaurin:big-data pat$ ls -al b/ssvd/Q-job/ >> total 72 >> drwxr-xr-x 10 pat staff 340 Aug 31 13:35 . >> drwxr-xr-x 4 pat staff 136 Aug 31 13:35 .. >> -rw-r--r-- 1 pat staff 80 Aug 31 13:35 .QHat-m-00000.crc >> -rw-r--r-- 1 pat staff 28 Aug 31 13:35 .R-m-00000.crc >> -rw-r--r-- 1 pat staff 8 Aug 31 13:35 ._SUCCESS.crc >> -rw-r--r-- 1 pat staff 12 Aug 31 13:35 .part-m-00000.deflate.crc >> -rwxrwxrwx 1 pat staff 9154 Aug 31 13:35 QHat-m-00000 >> -rwxrwxrwx 1 pat staff 2061 Aug 31 13:35 R-m-00000 >> -rwxrwxrwx 1 pat staff 0 Aug 31 13:35 _SUCCESS >> -rwxrwxrwx 1 pat staff 8 Aug 31 13:35 part-m-00000.deflate
