On Sep 11, 2010, at 10:11 PM, Drew Farris wrote: > Congratulations! > > What's the best way to send messages back to the caller of an EMR job, > using stderr instead of the log framework here?
It probably makes sense to have any command line errors, etc. go to stderr instead of logging the exception, but this may just be a relic of EMR and the way it configures logging. I will write up notes on the EMR wiki page. I just committed one minor fix that should help, namely printing out the OptionException when it occurs as part of the generic print options method. > > On Sat, Sep 11, 2010 at 9:32 PM, Grant Ingersoll <[email protected]> wrote: >> And indeed, running this via the Ruby CLI works as well. Woo hoo! >> >> -Grant >> >> On Sep 11, 2010, at 9:01 PM, Grant Ingersoll wrote: >> >>> >>> On Sep 11, 2010, at 8:02 PM, Grant Ingersoll wrote: >>> >>>> I've made a little bit of progress here, but not much. Here's what I ran: >>>> >>>> elastic-mapreduce -j <JOB> --jar >>>> s3n://news-vecs/mahout-core-0.4-SNAPSHOT.job --main-class >>>> org.apache.mahout.clustering.kmeans.KMeansDriver --arg --input --arg >>>> s3n://news-vecs/part-out.vec --arg --clusters --arg >>>> s3n://news-vecs/kmeans/clusters/ --arg >>> >>> >>>> --k >>> >>> Ugh. It's -k, not --k. >>> >>> So, this bit of code could likely be more useful: >>> } catch (IllegalArgumentException e) { >>> log.error(e.getMessage()); >>> CommandLineUtil.printHelpWithGenericOptions(group); >>> return null; >>> } >>> >>> Since, at least on EMR, the logs tend to get buried and it writes it out to >>> syslog, not stderr or stdout. >>> >>> I have it running now by logging into the EMR instance using SSH and then I >>> also specifically uploaded my Vector file to HDFS by hand. In other words, >>> I'm not using the remote Ruby CLI just yet. >>> >>> Progress. Sigh. >>> >>> -Grant >> >> -------------------------- >> Grant Ingersoll >> http://lucenerevolution.org Apache Lucene/Solr Conference, Boston Oct 7-8 >> >> -------------------------- Grant Ingersoll http://lucenerevolution.org Apache Lucene/Solr Conference, Boston Oct 7-8
