Re: Installing PyLucene

[Andi Vajda]

> On Jan 12, 2017, at 03:47, Jan Høydahl  wrote:
> 
> 
>>> The (sub)project will survive only to the extent that its current users 
>>> invest in it.
>> so is this different to how the main (Java)Lucene project works? just 
>> curious …
> 
> This is the case for all ASF projects and sub projects. If a project is 
> unable to produce
> new releases, usually due to too few developers, or loss of interest, or lack 
> of users,
> the project will end up in the Attic. For a TLP it would be the Board 
> intervening, while
> for sub projects I believe it is the TLPs responsibility to make sure things 
> are healthy.
> 
>>> For an ASF Open Source Project, the only thing that is required to get 
>>> going is user/developer
>>> involvement and teamwork. While Andi started the project due to needs at 
>>> the time, and became
>>> a committer, he is no longer an active user, so perhaps time has come for 
>>> other users to step ut and take
>>> responsibility.
>>> 
>> That’s first time I hear this (bad) news. So if Andi is no longer an active 
>> user - who is the maintainer of PyLucene/JCC?
> 
> I may have misunderstood myself, but it is quite normal for a commiter’s 
> availability to vary over the years,
> and for that reason it would be a big benefit for PyLucene to have at least 
> two other active devs voted in
> as committers. As I understand it, PyLucene does not require much effort to 
> keep up to date, except when
> there are major changes such as Py3 or porting tests etc.
> 
> I’m not a user myself, so I’m a bit on the sideline here, being on the PMC.
> 
>> We developed and provided the patch (which already took some time) for 
>> review and further adaption and really hope now for other users to step in. 
>> This is our current position for several reasons an after internal 
>> discussions which I cannot disclose here. Sorry. On the other hand, if we’d 
>> be the only user(s) interested/willing to push Python3 support (and 
>> Pylucene/JCC as a whole) then this project could not survive anyway I fear.
> 
> Absolutely, you have done “your share” and more, and that is what such a 
> project needs, and right
> now PyLucene needs more than one guy who can jump in and make Py3 happen.
> I guess what I was trying to get across is that if Andi will forever be the 
> one guy everyone depend on
> to get the smallest contribution landed in the codebase, it will not be 
> sustainable going forward, and
> probably not ideal for Andi nor the other users. 
> 
> So what would it take to bring one other developer up to a “pro" level, and 
> are there anyone who have such a desire?

Just as on the Lucene project: repeated sizable contributions, an understanding 
on how projects function in the Apache community, a commitment to work within 
it and a successful committership vote.

If someone were to donate time to make the JCC 3 patch work with trunk and 
submitted that and showed interest in the project in general, over time, not 
just doing a one time code drop, they'd be well qualified to become a committer.

Andi..

> 
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com

Re: Installing PyLucene

[Andi Vajda]

> On Jan 12, 2017, at 02:22, Thomas Koch  wrote:
> 
> Dear Jan, 
> thanks for your explanations.
> 
>> PyLucene is driven by its own community, and user involvement and 
>> contributions is a must.
> I’m quite familiar with community contributions - have already submitted code 
> and feedback, tested Pylucene releases, did provide support to other users 
> and voted several times for new releases.
> 
> 
>> The (sub)project will survive only to the extent that its current users 
>> invest in it.
> so is this different to how the main (Java)Lucene project works? just curious 
> …
> 
> 
>> For an ASF Open Source Project, the only thing that is required to get going 
>> is user/developer
>> involvement and teamwork. While Andi started the project due to needs at the 
>> time, and became
>> a committer, he is no longer an active user, so perhaps time has come for 
>> other users to step ut and take
>> responsibility.
>> 
> That’s first time I hear this (bad) news. So if Andi is no longer an active 
> user - who is the maintainer of PyLucene/JCC?

I am still the maintainer of this project. I am no longer using Lucene or 
PyLucene on a daily basis since 2011. I'm still happy to continue maintaining 
the project, though.

Andi..

> 
>> How “funding” would look like in the Python3 case is not so much sending 
>> money to the ASF,
>> but more for individual companies like your own, to sponsor (through 
>> developer time) the major
>> work on the patch, and driving it through to completion. Hopefully other 
>> users will contribute along
>> the way too.
> 
> 
> We developed and provided the patch (which already took some time) for review 
> and further adaption and really hope now for other users to step in. This is 
> our current position for several reasons an after internal discussions which 
> I cannot disclose here. Sorry. On the other hand, if we’d be the only user(s) 
> interested/willing to push Python3 support (and Pylucene/JCC as a whole) then 
> this project could not survive anyway I fear.
> 
> 
> best regards,
> 
> Thomas 
> —
>> Am 06.01.2017 um 12:32 schrieb Jan Høydahl :
>> 
>> Hi,
>> 
>>> I hope you didn’t get this wrong! We all appreciate the existence of 
>>> JCC/PyLucene and especially all the effort you’ve put into this.  
>> 
>> 
>> PyLucene is driven by its own community, and user involvement and 
>> contributions is a must.
>> The (sub)project will survive only to the extent that its current users 
>> invest in it.
>> 
>>> So if some funding is required to get this going …
>> 
>> For an ASF Open Source Project, the only thing that is required to get going 
>> is user/developer
>> involvement and teamwork. While Andi started the project due to needs at the 
>> time, and became
>> a committer, he is no longer an active user, so perhaps time has come for 
>> other users to step ut and take
>> responsibility.
>> 
>> How “funding” would look like in the Python3 case is not so much sending 
>> money to the ASF,
>> but more for individual companies like your own, to sponsor (through 
>> developer time) the major
>> work on the patch, and driving it through to completion. Hopefully other 
>> users will contribute along
>> the way too.
>> 
>> You will of course need help from experienced developers, but the ideal 
>> situation is that after
>> a couple of such patches that get committed, you (or the developer working 
>> on the code) will be nominated 
>> as committer and can continue developing PyLucene without the need for Andi 
>> or any other one individual.
>> 
>>> There has been some discussions about the future of PyLucene on this list 
>>> but I still didn't see any conclusion/decision
>> 
>> 
>> The discussion sparked some new development and a release, which is a 
>> success. So the decission I guess is to keep PyLucene alive and try to 
>> strengthen the community.
>> As long as the project continues to produce releases, it is (somewhat) alive.
>> If on the other hand another year or two goes by without another release, 
>> I’m sure the PMC will take action again.
> 

Re: Installing PyLucene

[Jan Høydahl]

>> The (sub)project will survive only to the extent that its current users 
>> invest in it.
> so is this different to how the main (Java)Lucene project works? just curious 
> …

This is the case for all ASF projects and sub projects. If a project is unable 
to produce
new releases, usually due to too few developers, or loss of interest, or lack 
of users,
the project will end up in the Attic. For a TLP it would be the Board 
intervening, while
for sub projects I believe it is the TLPs responsibility to make sure things 
are healthy.

>> For an ASF Open Source Project, the only thing that is required to get going 
>> is user/developer
>> involvement and teamwork. While Andi started the project due to needs at the 
>> time, and became
>> a committer, he is no longer an active user, so perhaps time has come for 
>> other users to step ut and take
>> responsibility.
>> 
> That’s first time I hear this (bad) news. So if Andi is no longer an active 
> user - who is the maintainer of PyLucene/JCC?

I may have misunderstood myself, but it is quite normal for a commiter’s 
availability to vary over the years,
and for that reason it would be a big benefit for PyLucene to have at least two 
other active devs voted in
as committers. As I understand it, PyLucene does not require much effort to 
keep up to date, except when
there are major changes such as Py3 or porting tests etc.

I’m not a user myself, so I’m a bit on the sideline here, being on the PMC.

> We developed and provided the patch (which already took some time) for review 
> and further adaption and really hope now for other users to step in. This is 
> our current position for several reasons an after internal discussions which 
> I cannot disclose here. Sorry. On the other hand, if we’d be the only user(s) 
> interested/willing to push Python3 support (and Pylucene/JCC as a whole) then 
> this project could not survive anyway I fear.

Absolutely, you have done “your share” and more, and that is what such a 
project needs, and right
now PyLucene needs more than one guy who can jump in and make Py3 happen.
I guess what I was trying to get across is that if Andi will forever be the one 
guy everyone depend on
to get the smallest contribution landed in the codebase, it will not be 
sustainable going forward, and
probably not ideal for Andi nor the other users. 

So what would it take to bring one other developer up to a “pro" level, and are 
there anyone who have such a desire?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

Re: Installing PyLucene

[Thomas Koch]

Dear Andi,

many thanks for your review of the patch and helpful comments.

As mentioned in my previous mail to the list I’m afraid to let you know that we 
currently cannot put more effort into this task. This may change in the future 
of course.

However, with „funding“ my idea was to look for open source funding such as 
provided by the Python Software Foundation (e.g.):
https://www.python.org/psf/grants

That’s probably worth a try - especially because the PSF is well known, has 
some sponsors and also receives donations by companies using Python and Python 
related projects on regular basis…

> Thank you for the work done so far, it's looking really good but it needs to
> be refreshed to JCC/trunk and latest Python 3 to minimize work on my side.

Ok. Maybe ASF could contact PSF and ask for such a grant that could be uses to 
>hire back the contractor who did the JCC Python 3 port originally and have
>him/her refresh it for the latest JCC on trunk
? (If he/she is willing to do so … that’s another question).

Just thinking about ways to get this forward. I’m afraid I cannot provide more 
support currently ,-(


best regards,

Thomas 
—
> Am 06.01.2017 um 22:01 schrieb Andi Vajda :
> 
> 
> I now took a look at the python 3 patches you sent a link to in an earlier 
> message and here is the gist of my thoughts:
>  - Moving the Python 3 is desirable but what about Python 2 support today
>in 2017 ? I have no desire to support both for PyLucene manually. If,
>somehow, there can be two versions of JCC, one for Python 2, one for
>Python 3 and the PyLucene tests can be 2to3'd automatically, then the
>Python 3 support idea looks more attractive already. Supporting two
>versions of JCC is fine until 2020.
> 
>  - The JCC patches look very reasonable but should be updated to the latest
>Python 3. In particular, the internal Python 3 string representation was
>changed again after 3.2 (?) and has clever optimizations possible based
>on the internal byte size of characters chosen by Python (internally)
>for each string, based on the range of the characters used in the string.
>This makes it possible to often just copy chars from Python to Java.
>I just did a rewrite for this in PyICU (another long
>term project of mine, https://github.com/ovalhub/pyicu/ 
> ) and the Python 3
>string story got much cleaner post 3.2 (at least more
>understandable). Lots of bugs with long unicode chars (forgot the proper
>term, sorry) got fixed along the way (emoticon support, yay).
> 
>So, if you're prepared to fund this effort, it might be best to hire
>back the contractor who did the JCC Python 3 port originally and have
>him/her refresh it for the latest JCC on trunk (not too many changes
>happened in the past few years) and to the use the Python internal string
>APIs that appeared post Python 3.2. The ones in use in the patch are
>deprecated already. I love it that we'd then shed _all_ backwards
>compatibility baggage in JCC going forward in Python 3.x, x >= 6.
> 
>If you get the JCC/Python3 patches into a shape where I can apply them to
>trunk without trouble and using the latest CPython string APIs:
>   https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AsUCS4 
> 
>and related (PyUnicode_KIND, etc...)
>then there is a good chance that PyLucene/JCC would be fully supported
>with Python 3.x, x >= 6.
> 
>  - The PyLucene patches should probably be redone so that they can be
>automated with 2to3. If we get JCC in shape, I can take care of the rest.
> 
> Thank you for the work done so far, it's looking really good but it needs to
> be refreshed to JCC/trunk and latest Python 3 to minimize work on my side.
> 
> Andi..

Re: Installing PyLucene

[Thomas Koch]

Dear Jan, 
thanks for your explanations.

> PyLucene is driven by its own community, and user involvement and 
> contributions is a must.
I’m quite familiar with community contributions - have already submitted code 
and feedback, tested Pylucene releases, did provide support to other users and 
voted several times for new releases.


> The (sub)project will survive only to the extent that its current users 
> invest in it.
so is this different to how the main (Java)Lucene project works? just curious …


> For an ASF Open Source Project, the only thing that is required to get going 
> is user/developer
> involvement and teamwork. While Andi started the project due to needs at the 
> time, and became
> a committer, he is no longer an active user, so perhaps time has come for 
> other users to step ut and take
> responsibility.
> 
That’s first time I hear this (bad) news. So if Andi is no longer an active 
user - who is the maintainer of PyLucene/JCC?

> How “funding” would look like in the Python3 case is not so much sending 
> money to the ASF,
> but more for individual companies like your own, to sponsor (through 
> developer time) the major
> work on the patch, and driving it through to completion. Hopefully other 
> users will contribute along
> the way too.


We developed and provided the patch (which already took some time) for review 
and further adaption and really hope now for other users to step in. This is 
our current position for several reasons an after internal discussions which I 
cannot disclose here. Sorry. On the other hand, if we’d be the only user(s) 
interested/willing to push Python3 support (and Pylucene/JCC as a whole) then 
this project could not survive anyway I fear.


best regards,

Thomas 
—
> Am 06.01.2017 um 12:32 schrieb Jan Høydahl :
> 
> Hi,
> 
>> I hope you didn’t get this wrong! We all appreciate the existence of 
>> JCC/PyLucene and especially all the effort you’ve put into this.  
> 
> 
> PyLucene is driven by its own community, and user involvement and 
> contributions is a must.
> The (sub)project will survive only to the extent that its current users 
> invest in it.
> 
>> So if some funding is required to get this going …
> 
> For an ASF Open Source Project, the only thing that is required to get going 
> is user/developer
> involvement and teamwork. While Andi started the project due to needs at the 
> time, and became
> a committer, he is no longer an active user, so perhaps time has come for 
> other users to step ut and take
> responsibility.
> 
> How “funding” would look like in the Python3 case is not so much sending 
> money to the ASF,
> but more for individual companies like your own, to sponsor (through 
> developer time) the major
> work on the patch, and driving it through to completion. Hopefully other 
> users will contribute along
> the way too.
> 
> You will of course need help from experienced developers, but the ideal 
> situation is that after
> a couple of such patches that get committed, you (or the developer working on 
> the code) will be nominated 
> as committer and can continue developing PyLucene without the need for Andi 
> or any other one individual.
> 
>> There has been some discussions about the future of PyLucene on this list 
>> but I still didn't see any conclusion/decision
> 
> 
> The discussion sparked some new development and a release, which is a 
> success. So the decission I guess is to keep PyLucene alive and try to 
> strengthen the community.
> As long as the project continues to produce releases, it is (somewhat) alive.
> If on the other hand another year or two goes by without another release, I’m 
> sure the PMC will take action again.

Re: Installing PyLucene

[Thomas Koch]

Dear Petrus, 
thanks for your feedback - good to know that it’s useful for you application. 
And at least one more user who asks for Python3 Support in JCC ,-)

best regards,
Thomas 
—
> Am 11.01.2017 um 23:14 schrieb Petrus Hyvönen :
> 
> Dear Thomas,
> 
> Thanks for the patch files, I am not using pylucene so I have not tried that 
> part, I am wrapping another java library (orekit). I have now built a JCC on 
> your JCC patch applied on JCCv2.22. The test suite and application I have for 
> my project runs fine with this patch under python 3, on mac, linux and 
> windows, I saw no deviations in behavior of the library compared to v2.7. 
> This is by far not a very deep test suite, but very good news.
> 
> With Best Regards
> /Petrus
> 
> 
> 
>> On 06 Jan 2017, at 10:07 , Thomas Koch > > wrote:
>> 
>> 
>>> Am 04.01.2017 um 10:59 schrieb Petrus Hyvönen >> >:
>>> 
>>> Dear Thomas,
>>> 
>>> I would be very interested in a python 3 port of JCC. I am not a very 
>>> skilled developer, looked at starting a development based on the old 
>>> python-3 version but it's beyond my current skills.
>>> 
>>> I would be happy to help and test and review the JCC patches, I think your 
>>> patches would be a valuable contribution to JCC.
>>> 
>>> With Best Regards
>>> /Petrus
>>> 
>> 
>> Dear Petrus,
>> the "patch" is available in this gist
>> https://gist.github.com/tommykoch/8b3cec18d6f3650d05d235c46ba3b055 
>> 
>> 
>> It consists of 3 files:
>>  - one build script (build.py)
>>  - a patch file for jcc (pylucene_3_6-jcc-P3.patch)
>>  - a patch file for pylucene3.6 (pylucene_3_6-P3.patch)
>>   
>> in our environment the 3 files are located in 
>>  /tools/build/pylucene
>> and the output (i.e. patched Pylucene36) is created in
>>  /build
>>  
>> if you create this structure you may simply run
>> $ cd tools/build/pylucene
>> $ build.py build
>> 
>> which will checkout Pylucene36 from SVN (in /build), apply the patch 
>> files and build JCC and PyLucene.
>> The Script is intended to run on Linux (or MacOS) - a diff binary and svn 
>> installation is required.
>> 
>> also easy_install(3) is needed to install the generated egg files -
>> besides the usual requirements for a PyLucene build (python3.4+, Java, gcc 
>> ...)
>> 
>> IMPORTANT NOTES:
>> - the patch is "one way" i.e. the patched Pylucene36 won't work with Py27 
>> anymore!
>> - the patch is for the old (and outdated) PyLucene36 version only - it has 
>> not yet been tested/ported to newer versions of JCC/Pylucene
>> - the patched JCC has only been used/tested with PyLucene - not with any 
>> other Java library
>> 
>> A review of a core developer of JCC/Pylucene would also be good - I’ll ask 
>> Andi what he thinks about this suggestion.
>> 
>> best regards,
>> 
>> Thomas
>