Hi Kenneth
Thank you for this detailed answer. I'll try to make it work using
m_gram functionnality, since the features I'm computing only make
sense for n-gram models. I let you know if it worked.
Thanks again!
Sylvain
On Tuesday 07 February 2012 10:17:26 Kenneth Heafield wrote:
> Hi,
>
> Moses server wants to keep one copy of a language model even if it
has
> different feature weights. The problem is that
StatefulFeatureFunction
> knows its own weight (a design I disagree with), so it can't just keep
> one StatefulFeatureFunction.
>
> LanguageModel is a child of StatefulFeatureFunction. All language
> models are child classes of LanguageModel.
>
> For KenLM, the object hierarchy looks like:
>
> ScoreProducer < FeatureFunction < StatefulFeatureFunction <
> LanguageModel < LanguageModelKen
>
> and LanguageModelKen has boost::shared_ptr<lm::ngram::Model>
m_ngram.
> There is no LanguageModelImplementation intermediary any more
but
> m_ngram provides similar functionality with a different interface.
>
> For all other language models, the object hierarchy is:
>
> ScoreProducer < FeatureFunction < StatefulFeatureFunction <
> LanguageModel < LMRefCount
>
> LMRefCount's sole purpose is to delegate to its member
> boost::shared_ptr<LanguageModelImplementation> m_impl
> thereby passing calls to the LanguageModelImplementation class.
>
> LanguageModelImplementation implements CalcScore, Evaluate, and
> EvaluateChart. These call virtual methods. SRILM's hierarchy is
>
> LanguageModelImplementation < LanguageModelSingleFactor <
> LanguageModelPointerState < LanguageModelSRI
>
> The interaction between these is somewhat messy and there's some
old
> code in there. I have simple wrappers for SRILM, IRSTLM, and RandLM
> that implement the same interface as lm::ngram::Model (except for
> ExtendLeft) and was considering making these wrappers template
arguments
> to LanguageModelKen (which would then be renamed). But it also
makes
> people suspect foul play if I break anything in the process.
>
> Kenneth
>
> On 02/07/2012 09:03 AM, Barry Haddow wrote:
> > Hi Sylvain
> >
> > I've updated the doxygen docs, if that helps,
> >
> > cheers - Barry
> >
> > On Tuesday 07 February 2012 12:24:07 Sylvain Raybaud wrote:
> >> Hi, List
> >>
> >> I've been digging into this but I'm still clueless... I really need to
> >> get
> >>
> >> the LanguageModelImplementation object, but nowhere in
Base.h/cpp or in
> >> specific implementations like Ken.h/cpp could I find where it
refers to
> >>
> >> LanguageModelImplementation. Is it still used at all?
> >>
> >> By the way, I think mosesserver has got the same problem since
its
> >> implementation uses the same methods I used.
> >>
> >> regards,
> >>
> >> Sylvain
> >>
> >> On Monday 06 February 2012 13:20:07 Sylvain Raybaud wrote:
> >>> Hello
> >>>
> >>> I'm trying to update a software that makes use of Moses API. It
uses
> >>> (among others) Moses' LanguageModel and
LanguageModelImplementation
> >>> classes. It used to call method GetLMImplementation of an
object of type
> >>> LanguageModel, in order to use methods NewState and
GetValueForgotState
> >>> of class LanguageModelImplementation. That was in august.
> >>> Unfortunatelly,
> >>> now I've pulled a new version of Moses from git, and method
> >>> GetLMImplementation seems to have vanished. Methods
NewState and
> >>> GetValueForgotState seems to still be implemented in class
> >>>
> >>> LanguageModelImplementation. Now I can get the LM using:
> >>> TranslationSystem ts =
> >>>
> >>>
StaticData::Instance().GetTranslationSystem(std::string("default"));
> >>> LMList list = ts.GetLanguageModels();
> >>>
> >>> LanguageModel *lm = *(list.begin());
> >>>
> >>> But I could not find a way to get the
LanguageModelImplementation object
> >>> in order to call aforementioned method. Do you have an idea of
what I
> >>> should change in the code?
> >>>
> >>> By the way, documentation on
> >>> http://www.statmt.org/moses/html/hierarchy.html seems to be
obsolete, as
> >>> it seems to be lacking many files, classes and functions.
> >>>
> >>> regards,
> >>>
> >>> Sylvain
> >>> _______________________________________________
> >>> Moses-support mailing list
> >>> Moses-support@mit.edu
> >>> http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> > --
> > Barry Haddow
> > University of Edinburgh
> > +44 (0) 131 651 3173
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
