[jira] [Commented] (MAHOUT-1200) Mahout tests depend on writing to /tmp/hadoop-$user
[ https://issues.apache.org/jira/browse/MAHOUT-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672307#comment-13672307 ] Hudson commented on MAHOUT-1200: Integrated in Mahout-Quality #2027 (See [https://builds.apache.org/job/Mahout-Quality/2027/]) MAHOUT-1200 - Mahout tests depend on writing to /tmp/hadoop-$user Change hadoop dependent tests to work out of a directory that is not under /tmp (Revision 1488627) Result = SUCCESS isabel : Files : * /mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/hadoop/item/RecommenderJobTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/cf/taste/hadoop/similarity/item/ItemSimilarityJobTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/classifier/naivebayes/NaiveBayesTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/canopy/TestCanopyCreation.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/classify/ClusterClassificationDriverTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/dirichlet/TestDirichletClustering.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/dirichlet/TestMapReduce.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/fuzzykmeans/TestFuzzyKmeansClustering.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/iterator/TestClusterClassifier.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/kmeans/TestKmeansClustering.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/lda/cvb/TestCVBModelTrainer.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/meanshift/TestMeanShift.java * /mahout/trunk/core/src/test/java/org/apache/mahout/clustering/minhash/TestMinHashClustering.java * /mahout/trunk/core/src/test/java/org/apache/mahout/common/MahoutTestCase.java * /mahout/trunk/core/src/test/java/org/apache/mahout/fpm/pfpgrowth/FPGrowthTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/fpm/pfpgrowth/PFPGrowthRetailDataTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/fpm/pfpgrowth/PFPGrowthTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/TestDistributedRowMatrix.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/decomposer/TestDistributedLanczosSolver.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/decomposer/TestDistributedLanczosSolverCLI.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/similarity/TestVectorDistanceSimilarityJob.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/similarity/cooccurrence/RowSimilarityJobTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/solver/TestDistributedConjugateGradientSolver.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/solver/TestDistributedConjugateGradientSolverCLI.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/hadoop/stats/BasicStatsTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/stats/entropy/ConditionalEntropyTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/stats/entropy/EntropyTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/stats/entropy/InformationGainRatioTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/math/stats/entropy/InformationGainTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/vectorizer/DictionaryVectorizerTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/vectorizer/DocumentProcessorTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/vectorizer/EncodedVectorsFromSequenceFilesTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/vectorizer/HighDFWordsPrunerTest.java * /mahout/trunk/core/src/test/java/org/apache/mahout/vectorizer/SparseVectorsFromSequenceFilesTest.java * /mahout/trunk/integration/src/main/java/org/apache/mahout/utils/SplitInput.java * /mahout/trunk/integration/src/test/java/org/apache/mahout/clustering/TestClusterDumper.java * /mahout/trunk/integration/src/test/java/org/apache/mahout/clustering/TestClusterEvaluator.java * /mahout/trunk/integration/src/test/java/org/apache/mahout/utils/SplitInputTest.java * /mahout/trunk/math/src/test/java/org/apache/mahout/math/MahoutTestCase.java * /mahout/trunk/pom.xml Mahout tests depend on writing to /tmp/hadoop-$user --- Key: MAHOUT-1200 URL: https://issues.apache.org/jira/browse/MAHOUT-1200 Project: Mahout Issue Type: Bug Components: build Affects Versions: 0.7 Reporter: Isabel Drost-Fromm Fix For: 0.8 Attachments: MAHOUT-1200.patch, MAHOUT-1200.patch, MAHOUT-1200.patch Running the Mahout test suite creates the temp directory
[jira] [Commented] (MAHOUT-1200) Mahout tests depend on writing to /tmp/hadoop-$user
[ https://issues.apache.org/jira/browse/MAHOUT-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13647377#comment-13647377 ] Dawid Weiss commented on MAHOUT-1200: - Another benefit is that if tests are run on a CI system such as jenkins, failed/ interrupted builds don't add up and pollute the /tmp space. This also has a drawback because another build wipes out any intermediate data but Jenkins (and other such systems) can be set up to archive selected artefacts on failed builds so I think this is minor). Mahout tests depend on writing to /tmp/hadoop-$user --- Key: MAHOUT-1200 URL: https://issues.apache.org/jira/browse/MAHOUT-1200 Project: Mahout Issue Type: Bug Components: build Affects Versions: 0.7 Reporter: Isabel Drost-Fromm Attachments: MAHOUT-1200.patch Running the Mahout test suite creates the temp directory /tmp/hadoop-$user which is used by all Hadoop related tests that pull up a local cluster. The directory is not removed after running the tests. In particular when running multiple tests in parallel on the same machine as the same user this can lead to problems. To re-produce issue the following commands prior to running the full test suite: mkdir /tmp/hadoop-$USER chmod 000 /tmp/hadoop-$USER mvn test -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1200) Mahout tests depend on writing to /tmp/hadoop-$user
[ https://issues.apache.org/jira/browse/MAHOUT-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13646424#comment-13646424 ] Isabel Drost commented on MAHOUT-1200: -- I tried to loosly follow the ideas in http://linux.die.net/man/3/mktemp Mahout tests depend on writing to /tmp/hadoop-$user --- Key: MAHOUT-1200 URL: https://issues.apache.org/jira/browse/MAHOUT-1200 Project: Mahout Issue Type: Bug Components: build Affects Versions: 0.7 Reporter: Isabel Drost Attachments: MAHOUT-1200.patch Running the Mahout test suite creates the temp directory /tmp/hadoop-$user which is used by all Hadoop related tests that pull up a local cluster. The directory is not removed after running the tests. In particular when running multiple tests in parallel on the same machine as the same user this can lead to problems. To re-produce issue the following commands prior to running the full test suite: mkdir /tmp/hadoop-$USER chmod 000 /tmp/hadoop-$USER mvn test -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1200) Mahout tests depend on writing to /tmp/hadoop-$user
[ https://issues.apache.org/jira/browse/MAHOUT-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13646422#comment-13646422 ] Isabel Drost commented on MAHOUT-1200: -- I'm happy with any naming schema, for add long add we keep the random number in there add otherwise we run into issues when more than one suite is running (potentially on a continuous integration server) or when many users run the test sit (e.g. remember the trouble we had on people.Apache.org Mahout tests depend on writing to /tmp/hadoop-$user --- Key: MAHOUT-1200 URL: https://issues.apache.org/jira/browse/MAHOUT-1200 Project: Mahout Issue Type: Bug Components: build Affects Versions: 0.7 Reporter: Isabel Drost Attachments: MAHOUT-1200.patch Running the Mahout test suite creates the temp directory /tmp/hadoop-$user which is used by all Hadoop related tests that pull up a local cluster. The directory is not removed after running the tests. In particular when running multiple tests in parallel on the same machine as the same user this can lead to problems. To re-produce issue the following commands prior to running the full test suite: mkdir /tmp/hadoop-$USER chmod 000 /tmp/hadoop-$USER mvn test -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1200) Mahout tests depend on writing to /tmp/hadoop-$user
[ https://issues.apache.org/jira/browse/MAHOUT-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13646484#comment-13646484 ] Isabel Drost commented on MAHOUT-1200: -- As suggested by Dawid one other option could be to simply create all test temp files not in /tmp but in the cwd - preferably in the mvn target directories to have mvn clean delete all build related data even in case tests were stopped by hitting ctrl-c What do you think? Mahout tests depend on writing to /tmp/hadoop-$user --- Key: MAHOUT-1200 URL: https://issues.apache.org/jira/browse/MAHOUT-1200 Project: Mahout Issue Type: Bug Components: build Affects Versions: 0.7 Reporter: Isabel Drost Attachments: MAHOUT-1200.patch Running the Mahout test suite creates the temp directory /tmp/hadoop-$user which is used by all Hadoop related tests that pull up a local cluster. The directory is not removed after running the tests. In particular when running multiple tests in parallel on the same machine as the same user this can lead to problems. To re-produce issue the following commands prior to running the full test suite: mkdir /tmp/hadoop-$USER chmod 000 /tmp/hadoop-$USER mvn test -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1200) Mahout tests depend on writing to /tmp/hadoop-$user
[ https://issues.apache.org/jira/browse/MAHOUT-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13646624#comment-13646624 ] Ted Dunning commented on MAHOUT-1200: - That sounds like an excellent idea. Ensuring that directories in /tmp get cleaned up is hard. Mahout tests depend on writing to /tmp/hadoop-$user --- Key: MAHOUT-1200 URL: https://issues.apache.org/jira/browse/MAHOUT-1200 Project: Mahout Issue Type: Bug Components: build Affects Versions: 0.7 Reporter: Isabel Drost-Fromm Attachments: MAHOUT-1200.patch Running the Mahout test suite creates the temp directory /tmp/hadoop-$user which is used by all Hadoop related tests that pull up a local cluster. The directory is not removed after running the tests. In particular when running multiple tests in parallel on the same machine as the same user this can lead to problems. To re-produce issue the following commands prior to running the full test suite: mkdir /tmp/hadoop-$USER chmod 000 /tmp/hadoop-$USER mvn test -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1200) Mahout tests depend on writing to /tmp/hadoop-$user
[ https://issues.apache.org/jira/browse/MAHOUT-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13645969#comment-13645969 ] Robin Anil commented on MAHOUT-1200: Can't you create a new subdirectory for each qualified class name /tmp/mahout-temp/org.apache.mahout.clustering.ClusteringTest/ ? Mahout tests depend on writing to /tmp/hadoop-$user --- Key: MAHOUT-1200 URL: https://issues.apache.org/jira/browse/MAHOUT-1200 Project: Mahout Issue Type: Bug Components: build Affects Versions: 0.7 Reporter: Isabel Drost Attachments: MAHOUT-1200.patch Running the Mahout test suite creates the temp directory /tmp/hadoop-$user which is used by all Hadoop related tests that pull up a local cluster. The directory is not removed after running the tests. In particular when running multiple tests in parallel on the same machine as the same user this can lead to problems. To re-produce issue the following commands prior to running the full test suite: mkdir /tmp/hadoop-$USER chmod 000 /tmp/hadoop-$USER mvn test -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
