DB Tsai created SPARK-60254:
-------------------------------
Summary: Replace Breeze delegation in vector and matrix primitives
Key: SPARK-60254
URL: https://issues.apache.org/jira/browse/SPARK-60254
Project: Spark
Issue Type: Improvement
Components: MLlib
Affects Versions: 5.0.0
Reporter: DB Tsai
Spark's local and legacy vector/matrix implementations still delegate basic
operations to Breeze. These operations can be implemented directly using
Spark's existing dense and sparse storage.
Replace Breeze delegation for vector element access, sparse CSC/CSR layout
conversion, matrix equality and sparse hashing, and matrix formatting. Share
the matrix implementations with legacy MLlib through its existing ML adapters.
Preserve compatibility for transposed matrices, explicitly stored zeros, NaN
and signed zero, output truncation, and locale-sensitive formatting.
This is a first step toward reducing Spark's Breeze dependency. Breeze remains
a runtime dependency for algorithm consumers, and the existing
asBreeze/fromBreeze adapters remain unchanged. Removing the dependency itself
is outside this issue's scope.
Add regression coverage for sparse layout conversion, equality and hashing
parity, and formatting parity in both local and legacy linear algebra suites.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
