Suggestion: enhance examples/bin/classify-20newsgroups.sh to allow using this to generate the model, along with the online program.
Lance On Mon, Dec 12, 2011 at 4:06 AM, jirapos...@reviews.apache.org (Commented) (JIRA) <j...@apache.org> wrote: > > [ > https://issues.apache.org/jira/browse/MAHOUT-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167462#comment-13167462 > ] > > jirapos...@reviews.apache.org commented on MAHOUT-918: > ------------------------------------------------------ > > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java, > line 36 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line36> > bq. > > bq. > Needs a comment about how this works. > > Added comments. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java, > lines 67-75 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line67> > bq. > > bq. > This really need a comment. What is the purpose here? > > Added comments. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java, > lines 98-111 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line98> > bq. > > bq. > What is this intended to do? Why? > > Added comments. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java, > line 30 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63196#file63196line30> > bq. > > bq. > Typo. > bq. > > bq. > Also, this doesn't say how this works or why it is the way it is. > > Fixed the typo and added comments. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java, > line 32 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63196#file63196line32> > bq. > > bq. > Shouldn't there be a combiner as well? > > A combiner isn't needed because each map task submits one value overall. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java, > line 53 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63196#file63196line53> > bq. > > bq. > A comment here about what this weight is would be nice. Also, how > can a double be a key? That is tantamount to comparing doubles which is bad. > > Added comments. it is not the weight of the classifier but the weight of the > weighted average. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDDriver.java, > line 99 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63197#file63197line99> > bq. > > bq. > Where does the InterruptedException come from? > > It comes from runIteration function. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDDriver.java, > lines 110-111 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63197#file63197line110> > bq. > > bq. > Use brackets > > Added brackets. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionMapReduceTest.java, > line 35 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63198#file63198line35> > bq. > > bq. > Should not throw Exception > > Added IO Exception and Interrupted Exception. > > > bq. On 2011-12-08 07:04:49, Ted Dunning wrote: > bq. > > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java, > lines 53-56 > bq. > <https://reviews.apache.org/r/3072/diff/2/?file=63195#file63195line53> > bq. > > bq. > This is nearly duplicated code. The mapper and reducer should > share some code to avoid inconsistent defaults. > > Created a base class which shares the same initialization code. > > > - issei > > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/3072/#review3734 > ----------------------------------------------------------- > > > On 2011-12-12 11:51:59, issei yoshida wrote: > bq. > bq. ----------------------------------------------------------- > bq. This is an automatically generated e-mail. To reply, visit: > bq. https://reviews.apache.org/r/3072/ > bq. ----------------------------------------------------------- > bq. > bq. (Updated 2011-12-12 11:51:59) > bq. > bq. > bq. Review request for mahout. > bq. > bq. > bq. Summary > bq. ------- > bq. > bq. MAHOUT-918 Parallelized SGD in MapReduce > bq. > bq. > bq. This addresses bug MAHOUT-918. > bq. https://issues.apache.org/jira/browse/MAHOUT-918 > bq. > bq. > bq. Diffs > bq. ----- > bq. > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/PassiveAggressive.java > 1213193 > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionDriver.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionMapper.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionReducer.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionDriver.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionMapper.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionReducer.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveDriver.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapper.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveReducer.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDDriver.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDMapper.java > PRE-CREATION > bq. > trunk/core/src/main/java/org/apache/mahout/classifier/sgd/mapreduce/SGDReducer.java > PRE-CREATION > bq. > trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/AdaptiveLogisticRegressionMapReduceTest.java > PRE-CREATION > bq. > trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/LogisticRegressionMapReduceTest.java > PRE-CREATION > bq. > trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/PassiveAggressiveMapReduceTest.java > PRE-CREATION > bq. > trunk/core/src/test/java/org/apache/mahout/classifier/sgd/mapreduce/SGDMapReduceTest.java > PRE-CREATION > bq. > bq. Diff: https://reviews.apache.org/r/3072/diff > bq. > bq. > bq. Testing > bq. ------- > bq. > bq. > bq. Thanks, > bq. > bq. issei > bq. > bq. > > > >> Implement SGD based classifiers using MapReduce >> ----------------------------------------------- >> >> Key: MAHOUT-918 >> URL: https://issues.apache.org/jira/browse/MAHOUT-918 >> Project: Mahout >> Issue Type: New Feature >> Components: Classification >> Affects Versions: 0.6 >> Reporter: issei yoshida >> Attachments: MAHOUT-918.patch, design.pdf >> >> >> Implement SGD based classifiers (Logistic Regression, Adaptive Logistic >> regression and Passive-Aggressive) using MapReduce. >> They are implemented using Iterative Parameter Mixtures algorithm which is >> referred to in the following papers. >> http://research.google.com/pubs/pub36948.html >> http://aclweb.org/anthology-new/N/N10/N10-1069.pdf >> http://books.nips.cc/papers/files/nips22/NIPS2009_0345.pdf > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators: > https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa > For more information on JIRA, see: http://www.atlassian.com/software/jira > > -- Lance Norskog goks...@gmail.com
