On Dec 8, 2011, at 2:39 PM, Grant Ingersoll wrote:
>
> On Dec 8, 2011, at 2:23 PM, Grant Ingersoll wrote:
>
>> If I add parallel, fork always to the main surefire config, I get failures
>> all over the place for things like:
>> Failed tests:
>> testHebbianSolver(org.apache.mahout.math.decomposer.hebbian.TestHebbianSolver):
>> Error: {0.06146049974880152 too high! (for eigen 3)
>> consistency(org.apache.mahout.math.jet.random.NormalTest): offset=0.000
>> scale=1.000 Z = 8.2
>> consistency(org.apache.mahout.math.jet.random.ExponentialTest):
>> offset=0.000 scale=100.000 Z = 8.7
>>
>
> Check that, it seems each run can produce different failures, which leads me
> to believe we have some shared values in our tests
Random.getRandom() the culprit, perhaps?
>
>
>> All of these pass individually and when not in parallel for me.
>>
>> Here's my config:
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-surefire-plugin</artifactId>
>> <version>2.11</version>
>> <configuration>
>> <parallel>classes</parallel>
>> <forkMode>always</forkMode>
>> <perCoreThreadCount>true</perCoreThreadCount>
>> </configuration>
>> </plugin>
>>
>> Anyone else seeing that?
>>
>>
>> On Dec 8, 2011, at 1:53 PM, Dmitriy Lyubimov wrote:
>>
>>> SSVD actually runs a rather small test but it is a MR job in local
>>> mode, there's nothing to cut down there in terms of size (not much
>>> anyway). It's just what it takes to initialize and run all jobs (and
>>> since it is local, it is also single threaded, so it actually runs V
>>> and U jobs sequentially instead of parallel so it's even longer
>>> because of that (4 jobs stringed all in all).
>>>
>>> But i will take a look, although even if i reduce solution size, it
>>> will still likely not reduce running time by more than 20%.
>>>
>>> On Thu, Dec 8, 2011 at 5:42 AM, David Murgatroyd <[email protected]> wrote:
>>>>
>>>>
>>>>
>>>>
>>>> On Dec 8, 2011, at 8:36 AM, Grant Ingersoll <[email protected]> wrote:
>>>>
>>>>> MAHOUT-916 and 917 are attempts to address the running time of our tests.
>>>>> As Sean rightfully pointed out, there are probably opportunities to
>>>>> simply cut down the sizes of some of these tests w/o effecting there
>>>>> correctness. To that end, if people can take a look at:
>>>>> https://builds.apache.org/job/Mahout-Quality/1237/testReport/junit/
>>>>>
>>>>> You can get a sense as to which tests are taking a long time. The main
>>>>> culprits are:
>>>>> 1. Vectorizer
>>>>> 2. SSVD
>>>>> 3. K-Means
>>>>> 4. taste.hadoop.item
>>>>> 5. taste.hadoop.als
>>>>> 6. PFPGrowth
>>>>>
>>>>>
>>>>> -Grant
>>>>>
>>>>> --------------------------------------------
>>>>> Grant Ingersoll
>>>>> http://www.lucidimagination.com
>>>>>
>>>>>
>>>>>
>>
>> --------------------------------------------
>> Grant Ingersoll
>> http://www.lucidimagination.com
>>
>>
>>
>
> --------------------------------------------
> Grant Ingersoll
> http://www.lucidimagination.com
>
>
>
--------------------------------------------
Grant Ingersoll
http://www.lucidimagination.com
