guohao xiao created SPARK-27447:
-----------------------------------
Summary: Add collaborate filtering Explain API in SPARKML
Key: SPARK-27447
URL: https://issues.apache.org/jira/browse/SPARK-27447
Project: Spark
Issue Type: New Feature
Components: ML
Affects Versions: 2.5.0
Reporter: guohao xiao
Machine learning recommender systems have supercharged the online retail
environment by directly targeting what the customer wants. While customers are
getting better product recommendations than ever before, in the age of GDPR
there is growing concern about customer privacy and transparency with ML
models. Many are asking, just why am I receiving these recommendations? While
the current Implicit Collaborative Filtering (CF) algorithm in spark.ml is
great for generating recommendations at scale, its currently lacks any method
to explain why a particular customer is getting the recommendations they are
getting. In this talk, we demonstrate a way to expand collaborative filtering
so that the viewing history of a customer can be directly related to their
recommendations. Why were you recommended footwear? Well, 40% of this
recommendation came from browsing runners and 20% came from the shorts you
recently purchased. Turns out, rethinking of the linear algebra in the current
spark.ml CF implementation makes this possible. We show how this is done and
demonstrate its implemented as a new feature to spark.ml, expanding the API to
allow everyone to explain recommendations at scale and create a more
transparent ML future.
This project is going to present in Spark summit 2019:
https://databricks.com/sparkaisummit/north-america/sessions-single-2019?id=56
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
