zhengruifeng commented on PR #58848:
URL: https://github.com/apache/spark/pull/58848#issuecomment-5698347342
The trigger is narrower than all multinomial training, but it is reachable
during normal
initialization:
1. Without an initial model, multinomial intercepts are initialized as
`log1p(classCount) - mean(log1p(classCounts))`.
2. For balanced classes, every intercept is mathematically zero.
Floating-point evaluation often
produces bitwise `0.0`, although it can sometimes leave a very small
rounding residual.
3. The initial linear coefficients are zero, so centering does not change
these margin offsets.
4. If a partition produces multiple training blocks (the default block size
is 1 MB), the first
block replaces the reusable buffer contents with softmax multipliers. For
a bitwise-zero offset,
the old condition skipped resetting that class slice before the next
`gemm(..., beta = 1.0)`, so
stale multipliers were added to the next block's margins.
After the first optimizer update, offsets are continuous values and an exact
zero is generally
unlikely. Thus the bug is configuration-dependent and mostly affects the
initial loss/gradient
evaluation of balanced multinomial datasets rather than every iteration or
dataset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
