Seth Hendrickson created SPARK-21152: ----------------------------------------
Summary: Use level 3 BLAS operations in LogisticAggregator Key: SPARK-21152 URL: https://issues.apache.org/jira/browse/SPARK-21152 Project: Spark Issue Type: Sub-task Components: ML Affects Versions: 2.1.1 Reporter: Seth Hendrickson In logistic regression gradient update, we currently compute by each individual row. If we blocked the rows together, we can do a blocked gradient update which leverages the BLAS GEMM operation. On high dimensional dense datasets, I've observed ~10x speedups. The problem here, though, is that it likely won't improve the sparse case so we need to keep both implementations around, and this blocked algorithm will require caching a new dataset of type: {code} BlockInstance(label: Vector, weight: Vector, features: Matrix) {code} We have avoided caching anything beside the original dataset passed to train in the past because it adds memory overhead if the user has cached this original dataset for other reasons. Here, I'd like to discuss whether we think this patch would be worth the investment, given that it only improves a subset of the use cases. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org