Can someone help me to understand the usage of foreachActive function
introduced for the Vectors.
I am trying to understand its usage in MultivariateOnlineSummarizer class
for summary statistics.
sample.foreachActive { (index, value) =
if (value != 0.0) {
if (currMax(index)
The idea is to unify the code path for dense and sparse vector operations,
which makes the codebase easier to maintain. By handling (index, value)
tuples, you can let the foreachActive method take care of checking if the
vector is sparse or dense, and running a foreach over the values.
On Sun,
PS, we were using Breeze's activeIterator originally as you can see in
the old code, but we found there are overhead there, so we implement
our own implementation which results 4x faster. See
https://github.com/apache/spark/pull/3288 for detail.
Sincerely,
DB Tsai
