zhengruifeng opened a new pull request, #58276:
URL: https://github.com/apache/spark/pull/58276
### What changes were proposed in this pull request?
This PR converts the feed-forward model's weight vector to a dense array
once and shares that
array among the non-overlapping Breeze views used by its layer models.
It also adds a regression test that constructs a model from sparse weights
and verifies that all
affine layers share one dense backing array.
### Why are the changes needed?
MLP weights produced by normal training are typically dense, but `Vector`,
`initialWeights`, and
the persisted model format do not guarantee a dense representation.
`FeedForwardModel` previously
called `weights.toArray` inside the layer loop. Each affine layer retained
its resulting full-size
dense array even though it used only one slice, producing one complete
weight copy per affine
layer. Sparse weights also underwent the same full sparse-to-dense
conversion repeatedly.
Converting once preserves support for both dense and sparse inputs while
avoiding the repeated
allocations and retained copies.
### Does this PR introduce _any_ user-facing change?
No. Model results and APIs are unchanged; the implementation uses less
memory.
### How was this patch tested?
A regression test was added to `ANNSuite` using sparse model weights.
```
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 build/sbt \
'mllib/testOnly org.apache.spark.ml.ann.ANNSuite'
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
