Dear Moses Devs and Community, Sorry for the delayed response.
We've repackaged the MosesTokenizer Python code as a library and made it pip-able. https://github.com/alvations/sacremoses I hope that's okay with the Moses community and the license compliance is good with this now. Regards, Liling On Wed, Apr 11, 2018 at 1:41 AM, Matt Post <p...@cs.jhu.edu> wrote: > Seems worth a shot. I suggest contacting each of them with individual > emails until (and if) you get a “no”. > > matt (from my phone) > > Le 10 avr. 2018 à 19:26, liling tan <alvati...@gmail.com> a écrit : > > @Matt I'm not sure whether that'll work. > > > For tokenizer, that'll include: > > > - [image: @phikoehn] phikoehn <https://github.com/phikoehn> > - [image: @hieuhoang] hieuhoang <https://github.com/hieuhoang> > - [image: @bhaddow] bhaddow <https://github.com/bhaddow> > - [image: @jimregan] jimregan <https://github.com/jimregan> > - [image: @kpu] kpu <https://github.com/kpu> > - [image: @ugermann] ugermann <https://github.com/ugermann> > - [image: @pjwilliams] pjwilliams <https://github.com/pjwilliams> > - [image: @jgwinnup] jgwinnup <https://github.com/jgwinnup> > - [image: @mhuck] mhuck <https://github.com/mhuck> > - [image: @tofula] tofula <https://github.com/tofula> > - [image: @a455bcd9] a455bcd9 <https://github.com/a455bcd9> > > > And these for the detokenizer: > > - > [image: @phikoehn] phikoehn <https://github.com/phikoehn> > - [image: @flammie] flammie <https://github.com/flammie> > - [image: @hieuhoang] hieuhoang <https://github.com/hieuhoang> > - [image: @pjwilliams] pjwilliams <https://github.com/pjwilliams> > - [image: @bhaddow] bhaddow <https://github.com/bhaddow> > - [image: @alvations] alvations <https://github.com/alvations> > > Not sure if everyone agrees though. > > Regards, > Liling > > On Wed, Apr 11, 2018 at 12:39 AM, Matt Post <p...@cs.jhu.edu> wrote: > >> Liling—Would it work to get the permission of just those people who are >> in the commit log of the specific scripts you want to port? >> >> matt (from my phone) >> >> Le 10 avr. 2018 à 18:19, liling tan <alvati...@gmail.com> a écrit : >> >> Got it. >> >> So I think we'll just remove the MosesTokenizer and MosesDetokenizer >> function from NLTK and maybe create a PR to put it in >> mosesdecoder/scripts/tokenizer >> >> Thank you for the clarification! >> Liling >> >> On Wed, Apr 11, 2018 at 12:17 AM, Hieu Hoang <hieuho...@gmail.com> wrote: >> >>> Still the same problem - everyone owns Moses so you need everyone's >>> permission, not just mine. So no >>> >>> Hieu Hoang >>> http://moses-smt.org/ >>> >>> >>> On 10 April 2018 at 17:13, liling tan <alvati...@gmail.com> wrote: >>> >>>> I understand. >>>> >>>> Could we have permission that it's okay to derive work from Moses with >>>> respect to the (de-)tokenizer and possibly other scripts under an >>>> MIT/Apache tool? >>>> >>>> Legally it's a restriction but I think for what's it worth, having >>>> mutual agreement between the OSS is sufficient to still keep any port of >>>> LGPL work until someone starts to enforce legal actions and I think it's >>>> safe to back off to taking down these functionalities in the Apache/MIT >>>> code. >>>> >>>> Regards, >>>> Liling >>>> >>>> On Wed, Apr 11, 2018 at 12:09 AM, Hieu Hoang <hieuho...@gmail.com> >>>> wrote: >>>> >>>>> we can't change the license, or dual license it, without the agreement >>>>> of everyone who's contributed to Moses. Too much work >>>>> >>>>> Hieu Hoang >>>>> http://moses-smt.org/ >>>>> >>>>> >>>>> On 10 April 2018 at 15:47, liling tan <alvati...@gmail.com> wrote: >>>>> >>>>>> Dear Moses Dev, >>>>>> >>>>>> NLTK has a Python port of the word tokenizer in Moses. The tokenizer >>>>>> works well in Python and create a good synergy to bridge Python users to >>>>>> the code that Moses developers have spent years to hone. >>>>>> >>>>>> But it seemed to have hit a wall with some licensing issues. >>>>>> https://github.com/nltk/nltk/issues/2000 >>>>>> >>>>>> General port of LGPL code is considered derivative and is >>>>>> incompatible with Apache or MIT license. I understand that LGPL keeps >>>>>> derivative from being proprietary but it's a little less permissive than >>>>>> non-copyleft license like Apache and MIT licenses. >>>>>> >>>>>> Note that this licensing issue might also affect Marian which is MIT >>>>>> license and also incompatible with LGPL so although technically users can >>>>>> chain the code from different libraries, but Marian couldn't have any >>>>>> dependencies on the Moses components. (But we know do know that none of >>>>>> our >>>>>> models built with Marian would work without the Moses tokenizer which is >>>>>> in >>>>>> LGPL). >>>>>> >>>>>> Would there be a possibility to dual license the Moses repository >>>>>> with LGPL and Apache/BSD/MIT license. I'm not sure whether it's allowed >>>>>> to >>>>>> have dual licenses with LGPL and Apache/BSD/MIT license though. Might >>>>>> have >>>>>> to check with some proper legal personnel though. >>>>>> >>>>>> If dual license is not possible would it be possible relicense the >>>>>> code under BSD/Apache/MIT license? That way it's more permissive for >>>>>> derivatiive work? >>>>>> >>>>>> I think the last scenario is for NLTK to drop the Python port of >>>>>> Moses code entirely from Apache license repository but I think that'll >>>>>> remove the synergy between various OSS. >>>>>> >>>>>> Hope to hear from Moses devs soon! >>>>>> >>>>>> Regards, >>>>>> Liling >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> Moses-support@mit.edu >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>>> >>>>> >>>> >>> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > On Wed, Apr 11, 2018 at 1:41 AM, Matt Post <p...@cs.jhu.edu> wrote: > Seems worth a shot. I suggest contacting each of them with individual > emails until (and if) you get a “no”. > > matt (from my phone) > > Le 10 avr. 2018 à 19:26, liling tan <alvati...@gmail.com> a écrit : > > @Matt I'm not sure whether that'll work. > > > For tokenizer, that'll include: > > > - [image: @phikoehn] phikoehn <https://github.com/phikoehn> > - [image: @hieuhoang] hieuhoang <https://github.com/hieuhoang> > - [image: @bhaddow] bhaddow <https://github.com/bhaddow> > - [image: @jimregan] jimregan <https://github.com/jimregan> > - [image: @kpu] kpu <https://github.com/kpu> > - [image: @ugermann] ugermann <https://github.com/ugermann> > - [image: @pjwilliams] pjwilliams <https://github.com/pjwilliams> > - [image: @jgwinnup] jgwinnup <https://github.com/jgwinnup> > - [image: @mhuck] mhuck <https://github.com/mhuck> > - [image: @tofula] tofula <https://github.com/tofula> > - [image: @a455bcd9] a455bcd9 <https://github.com/a455bcd9> > > > And these for the detokenizer: > > - > [image: @phikoehn] phikoehn <https://github.com/phikoehn> > - [image: @flammie] flammie <https://github.com/flammie> > - [image: @hieuhoang] hieuhoang <https://github.com/hieuhoang> > - [image: @pjwilliams] pjwilliams <https://github.com/pjwilliams> > - [image: @bhaddow] bhaddow <https://github.com/bhaddow> > - [image: @alvations] alvations <https://github.com/alvations> > > Not sure if everyone agrees though. > > Regards, > Liling > > On Wed, Apr 11, 2018 at 12:39 AM, Matt Post <p...@cs.jhu.edu> wrote: > >> Liling—Would it work to get the permission of just those people who are >> in the commit log of the specific scripts you want to port? >> >> matt (from my phone) >> >> Le 10 avr. 2018 à 18:19, liling tan <alvati...@gmail.com> a écrit : >> >> Got it. >> >> So I think we'll just remove the MosesTokenizer and MosesDetokenizer >> function from NLTK and maybe create a PR to put it in >> mosesdecoder/scripts/tokenizer >> >> Thank you for the clarification! >> Liling >> >> On Wed, Apr 11, 2018 at 12:17 AM, Hieu Hoang <hieuho...@gmail.com> wrote: >> >>> Still the same problem - everyone owns Moses so you need everyone's >>> permission, not just mine. So no >>> >>> Hieu Hoang >>> http://moses-smt.org/ >>> >>> >>> On 10 April 2018 at 17:13, liling tan <alvati...@gmail.com> wrote: >>> >>>> I understand. >>>> >>>> Could we have permission that it's okay to derive work from Moses with >>>> respect to the (de-)tokenizer and possibly other scripts under an >>>> MIT/Apache tool? >>>> >>>> Legally it's a restriction but I think for what's it worth, having >>>> mutual agreement between the OSS is sufficient to still keep any port of >>>> LGPL work until someone starts to enforce legal actions and I think it's >>>> safe to back off to taking down these functionalities in the Apache/MIT >>>> code. >>>> >>>> Regards, >>>> Liling >>>> >>>> On Wed, Apr 11, 2018 at 12:09 AM, Hieu Hoang <hieuho...@gmail.com> >>>> wrote: >>>> >>>>> we can't change the license, or dual license it, without the agreement >>>>> of everyone who's contributed to Moses. Too much work >>>>> >>>>> Hieu Hoang >>>>> http://moses-smt.org/ >>>>> >>>>> >>>>> On 10 April 2018 at 15:47, liling tan <alvati...@gmail.com> wrote: >>>>> >>>>>> Dear Moses Dev, >>>>>> >>>>>> NLTK has a Python port of the word tokenizer in Moses. The tokenizer >>>>>> works well in Python and create a good synergy to bridge Python users to >>>>>> the code that Moses developers have spent years to hone. >>>>>> >>>>>> But it seemed to have hit a wall with some licensing issues. >>>>>> https://github.com/nltk/nltk/issues/2000 >>>>>> >>>>>> General port of LGPL code is considered derivative and is >>>>>> incompatible with Apache or MIT license. I understand that LGPL keeps >>>>>> derivative from being proprietary but it's a little less permissive than >>>>>> non-copyleft license like Apache and MIT licenses. >>>>>> >>>>>> Note that this licensing issue might also affect Marian which is MIT >>>>>> license and also incompatible with LGPL so although technically users can >>>>>> chain the code from different libraries, but Marian couldn't have any >>>>>> dependencies on the Moses components. (But we know do know that none of >>>>>> our >>>>>> models built with Marian would work without the Moses tokenizer which is >>>>>> in >>>>>> LGPL). >>>>>> >>>>>> Would there be a possibility to dual license the Moses repository >>>>>> with LGPL and Apache/BSD/MIT license. I'm not sure whether it's allowed >>>>>> to >>>>>> have dual licenses with LGPL and Apache/BSD/MIT license though. Might >>>>>> have >>>>>> to check with some proper legal personnel though. >>>>>> >>>>>> If dual license is not possible would it be possible relicense the >>>>>> code under BSD/Apache/MIT license? That way it's more permissive for >>>>>> derivatiive work? >>>>>> >>>>>> I think the last scenario is for NLTK to drop the Python port of >>>>>> Moses code entirely from Apache license repository but I think that'll >>>>>> remove the synergy between various OSS. >>>>>> >>>>>> Hope to hear from Moses devs soon! >>>>>> >>>>>> Regards, >>>>>> Liling >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> Moses-support@mit.edu >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>>> >>>>> >>>> >>> >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
