A/B testing it the typical way to compare the performance of 2 or more versions of an app. You should make sure that the versions only vary in 1 way to isolate what you are comparing. So channel some traffic to a page “without recommendations” and some amount of traffic to a page “with recommendations” and measure your conversion KPI for both cases. If your conversion is a sale or purchase you may have to split all users permanently over the test period. if you are only measuring CTR you may be able to split impressions since you don’t have to track users as a group all the way to a purchase. On Sep 26, 2016, at 5:45 AM, Gustavo Frederico <gustavo.freder...@thinkwrap.com> wrote:
Allow me to elaborate some more: there's a second question related to the original one. How to score a model in Production? Thanks On Fri, Sep 23, 2016 at 2:58 PM, Gustavo Frederico <gustavo.freder...@thinkwrap.com> wrote: > Considering PIO + the UR template, what are some of the ways to score > the model? I can think of some A/B testing but that would imply > relying on things outside the PIO + UR stack. Is there a way to > measure lift or "score the model"? Cross-validation with separate > testing set? > > Thanks > > Gustavo
