Re: Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
On 2011-06-02 18:26, Andi Vajda wrote: On Jun 2, 2011, at 3:10, Philippe Ombredannepombreda...@gmail.com wrote: On 2011-06-01 20:54, Roman Chyla wrote: Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. This is actually worse than I thought: not only the java location seems hardcoded in the shared object as a hard path to the libs folder, but also there is an implied dep on setuptools via pkg_resources So for now, you cannot even build on a jdk and deploy on a jre. If the solution to this is to remove the hardcoded paths and expect the dynamic linker to find the dependencies via some environment variable like LD_LIBRARY_PATH you'd be creating a security vulnerability. This is how I did it originally (years ago) and people complained about it so I switched to hardcoded paths for shared library dependencies wherever possible. Andi: could we come at least with a build option to enable this? I am sure there are bad ways and correct ways to address possible security issues as there are many packages that rely on this. And given the variety of *nix locations for a JVM and the various JVMs available we need some flexibility there imho. Leave it hardcoded by default allright and a bbuild flag to get it working otherwise if ones wants to build it this way? -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Re: Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
Hi Philippe, On Jun 4, 2011, at 9:42, Philippe Ombredanne pombreda...@gmail.com wrote: On 2011-06-02 18:26, Andi Vajda wrote: On Jun 2, 2011, at 3:10, Philippe Ombredannepombreda...@gmail.com wrote: On 2011-06-01 20:54, Roman Chyla wrote: Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. This is actually worse than I thought: not only the java location seems hardcoded in the shared object as a hard path to the libs folder, but also there is an implied dep on setuptools via pkg_resources So for now, you cannot even build on a jdk and deploy on a jre. If the solution to this is to remove the hardcoded paths and expect the dynamic linker to find the dependencies via some environment variable like LD_LIBRARY_PATH you'd be creating a security vulnerability. This is how I did it originally (years ago) and people complained about it so I switched to hardcoded paths for shared library dependencies wherever possible. Andi: could we come at least with a build option to enable this? The option already exists: when building jcc, you can pass it your own version of JCC_LFLAGS set as an env variable that overrides the defaults to whatever you like (see jcc's setup.py for more details). I am sure there are bad ways and correct ways to address possible security issues as there are many packages that rely on this. And given the variety of *nix locations for a JVM and the various JVMs available we need some flexibility there imho. Leave it hardcoded by default allright and a bbuild flag to get it working otherwise if ones wants to build it this way? If you are thinking of something different from the existing solution, patches are welcome. Maybe, all it needs is better documentation ? Thanks ! Andi.. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
On 2011-06-01 20:54, Roman Chyla wrote: Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. This is actually worse than I thought: not only the java location seems hardcoded in the shared object as a hard path to the libs folder, but also there is an implied dep on setuptools via pkg_resources So for now, you cannot even build on a jdk and deploy on a jre. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Re: Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
On Thu, 2 Jun 2011, Roman Chyla wrote: On Thu, Jun 2, 2011 at 12:26 PM, Andi Vajda va...@apache.org wrote: On Jun 2, 2011, at 3:10, Philippe Ombredanne pombreda...@gmail.com wrote: On 2011-06-01 20:54, Roman Chyla wrote: Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. This is actually worse than I thought: not only the java location seems hardcoded in the shared object as a hard path to the libs folder, but also there is an implied dep on setuptools via pkg_resources So for now, you cannot even build on a jdk and deploy on a jre. If the solution to this is to remove the hardcoded paths and expect the dynamic linker to find the dependencies via some environment variable like LD_LIBRARY_PATH you'd be creating a security vulnerability. I am not an expert on this, but i remember that LD_LIBRARY_PATH was not recommended (as it could break other libraries, if i remember well). So that's why I thought more about a 'more, standard' hardcoded locations. Or is there something else besides LD_LIBRARY_PATH and multiple hardcoded paths? Each Linux distribution has its own notion of more standard locations for java libraries. And then there are different java libraries for different java distributions too. Andi.. Roman This is how I did it originally (years ago) and people complained about it so I switched to hardcoded paths for shared library dependencies wherever possible. Andi.. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Re: Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
On Jun 2, 2011, at 3:10, Philippe Ombredanne pombreda...@gmail.com wrote: On 2011-06-01 20:54, Roman Chyla wrote: Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. This is actually worse than I thought: not only the java location seems hardcoded in the shared object as a hard path to the libs folder, but also there is an implied dep on setuptools via pkg_resources So for now, you cannot even build on a jdk and deploy on a jre. If the solution to this is to remove the hardcoded paths and expect the dynamic linker to find the dependencies via some environment variable like LD_LIBRARY_PATH you'd be creating a security vulnerability. This is how I did it originally (years ago) and people complained about it so I switched to hardcoded paths for shared library dependencies wherever possible. Andi.. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Re: Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
Andi Vajda va...@apache.org wrote: If the solution to this is to remove the hardcoded paths and expect the dynamic linker to find the dependencies via some environment variable like LD_LIBRARY_PATH you'd be creating a security vulnerability. This is how I did it originally (years ago) and people complained about it so I switched to hardcoded paths for shared library dependencies wherever possible. Which I for one appreciate! Bill
Re: Building is too difficult and release of a first pre-built egg
Hi Philippe, On Thu, Jun 2, 2011 at 5:54 AM, Philippe Ombredanne pombreda...@gmail.com wrote: On 2011-06-01 20:54, Roman Chyla wrote: I would build some other binaries and upload them, will you get me access? Done: I added you as a committer to http://code.google.com/a/apache-extras.org/p/pylucene-extra/ Thanks! I'll try to keep and post detailed logs for each build I do. I am planning to add some detailed egg building instructions too. I will also contact the dudes at: http://code.google.com/p/pylucene-win32-binary/ They are building windows eggs already I'll also do -- the project for which it is needed is this one: https://github.com/romanchyla/montysolr But I also need to build JCC and upload them. Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. Ah... good point... meaning this is bad... a build would not be java location independent then? This would be a major bummer to have the path to java hardcoded in the .so. You could commit the patches there if you have some? oh, I assumed that was not patcheable -- but maybe I was wrong; but what I certainly planned to do is to change each binary produced and set some standard paths. Any ideas of what would be the standard library paths for linux? Building pylucene/jcc is indeed difficult for newcomers. Indeed too hard imho. A big deterrent. Such that it does likely impair the project reach, growth and health. and it is a very wonderful project, i agree roman On Wed, Jun 1, 2011 at 10:54 AM, Philippe Ombredanne pombreda...@gmail.com wrote: Howdy! I think it is way too hard to build PyLucene for the mere mortals. Getting eggs is yet another level of difficulties I created an issue: https://issues.apache.org/jira/browse/PYLUCENE-10 and started an Apache extra project, releasing a first egg for the Linux 64/Python 2.5.2/Oracle JDK 1.5 combo http://code.google.com/a/apache-extras.org/p/pylucene-extra/downloads/list I hope that can help some folks. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Re: Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
On Thu, Jun 2, 2011 at 12:26 PM, Andi Vajda va...@apache.org wrote: On Jun 2, 2011, at 3:10, Philippe Ombredanne pombreda...@gmail.com wrote: On 2011-06-01 20:54, Roman Chyla wrote: Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. This is actually worse than I thought: not only the java location seems hardcoded in the shared object as a hard path to the libs folder, but also there is an implied dep on setuptools via pkg_resources So for now, you cannot even build on a jdk and deploy on a jre. If the solution to this is to remove the hardcoded paths and expect the dynamic linker to find the dependencies via some environment variable like LD_LIBRARY_PATH you'd be creating a security vulnerability. I am not an expert on this, but i remember that LD_LIBRARY_PATH was not recommended (as it could break other libraries, if i remember well). So that's why I thought more about a 'more, standard' hardcoded locations. Or is there something else besides LD_LIBRARY_PATH and multiple hardcoded paths? Roman This is how I did it originally (years ago) and people complained about it so I switched to hardcoded paths for shared library dependencies wherever possible. Andi.. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Re: Hardcoded java paths in shared objects [was:Re: Building is too difficult and release of a first pre-built egg]
On Thu, Jun 2, 2011 at 6:10 AM, Philippe Ombredanne pombreda...@gmail.com wrote: On 2011-06-01 20:54, Roman Chyla wrote: Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. This is actually worse than I thought: not only the java location seems hardcoded in the shared object as a hard path to the libs folder, but also there is an implied dep on setuptools via pkg_resources So for now, you cannot even build on a jdk and deploy on a jre. I am sorry, but I don't understand - what is the additional dependency hardcoded there? Thanks, Roman -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Re: Building is too difficult and release of a first pre-built egg
Hi Philippe, I would build some other binaries and upload them, will you get me access? But I also need to build JCC and upload them. Note that the location of the java that was used for the project built will be hardcoded inside the dynamic library, but I plan to change the header and set a few standard paths there. Building pylucene/jcc is indeed difficult for newcomers. Cheers, Roman On Wed, Jun 1, 2011 at 10:54 AM, Philippe Ombredanne pombreda...@gmail.com wrote: Howdy! I think it is way too hard to build PyLucene for the mere mortals. Getting eggs is yet another level of difficulties I created an issue: https://issues.apache.org/jira/browse/PYLUCENE-10 and started an Apache extra project, releasing a first egg for the Linux 64/Python 2.5.2/Oracle JDK 1.5 combo http://code.google.com/a/apache-extras.org/p/pylucene-extra/downloads/list I hope that can help some folks. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://twitter.com/pombr http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
Building is too difficult and release of a first pre-built egg
Howdy! I think it is way too hard to build PyLucene for the mere mortals. Getting eggs is yet another level of difficulties I created an issue: https://issues.apache.org/jira/browse/PYLUCENE-10 and started an Apache extra project, releasing a first egg for the Linux 64/Python 2.5.2/Oracle JDK 1.5 combo http://code.google.com/a/apache-extras.org/p/pylucene-extra/downloads/list I hope that can help some folks. -- Cordially Philippe philippe ombredanne | 1 650 799 0949 | pombredanne at nexb.com nexB - Open by Design (tm) - http://www.nexb.com http://twitter.com/pombr http://eclipse.org/atf - http://eclipse.org/soc - http://eclipse.org/vep http://drools.org/ - http://easyeclipse.org - http://phpeclipse.com
