[
https://issues.apache.org/jira/browse/IGNITE-10144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703391#comment-16703391
]
Artem Malykh commented on IGNITE-10144:
---------------------------------------
According to series of quick experiments, approach used now
{code:java}
upstream.sequential().flatMap(en -> Stream.generate(() ->
en).limit(poisson.sample()))
{code}
outperforms following approaches:
1. Using hash code for determinism instead of "sequential" and then using
parallel
{code:java}
PoissonWithUnderlyingRandomAccess p = new PoissonWithUnderlyingRandomAccess(
new Well19937c(123L),
0.3,
PoissonDistribution.DEFAULT_EPSILON,
PoissonDistribution.DEFAULT_MAX_ITERATIONS
upstream
.flatMap(en -> {
p.getRand().setSeed(1234L + en.hashCode());
return Stream.generate(() -> en).limit(p.sample());
}).parallel().count();{code}
2. Zipping upstream with indexes stream and then using index for determinism.
> Optimize bagging upstream transformer
> -------------------------------------
>
> Key: IGNITE-10144
> URL: https://issues.apache.org/jira/browse/IGNITE-10144
> Project: Ignite
> Issue Type: Improvement
> Components: ml
> Reporter: Artem Malykh
> Assignee: Artem Malykh
> Priority: Minor
> Fix For: 2.8
>
>
> For now BaggingUpstreamTransformer makes upstream sequential to make
> transformation deterministic. Maybe we should do it other way, for example
> use mapping of the form (entryIdx, en) -> Stream.generate(() -> en).limit(new
> PoissonDistribution(Well19937c(entryIdx + seed), ...).sample())
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
