Re: simple math/statistics problem using map reduce

2013-11-28 Thread Mark Hahn
Thanks. That makes sense. Using map/reduce was as much a curiosity as a practical requirement. Another way to monitor accuracy is to watch my progress indicator and see how close it is to the real time. On Thu, Nov 28, 2013 at 10:50 AM, Nitin Borwankar wrote: > Hi Mark, > > It may not be wort

Re: simple math/statistics problem using map reduce

2013-11-28 Thread Nitin Borwankar
Hi Mark, It may not be worth it to have a real time estimate of the coefficients A, B if the variance is very small. In other words if your collection of past videos cover most of the different kinds of videos you are likely to encounter then estimates of A, B are likely to be pretty robust and n

simple math/statistics problem using map reduce

2013-11-27 Thread Mark Hahn
I'm not an expert on statistics (and I'm lazy) so I thought I'd pose my problem here. Consider it a holiday mind exercise while avoiding relatives. I send customer-uploaded videos to Amazon Elastic Transcoder to generate a video for html5 consumption. It takes a few seconds up to tens of minutes