Re: Java ports: source vs. binary?
Kurt Miller wrote: I'm not sure if these things will make it for 4.0 yet. Let me know if I can help. -Fred
Re: Java ports: source vs. binary?
> PORTS WITH NATIVE DEPENDENCIES > == > > Ports that require native platform support are a different matter. > > Right now we have explicit ports dependencies on the Sun JDK tool chain > in the ports that are built from source. With respect to building packages from source the most compatiable JDK is selected (Sun's). I don't intend to change that or add flavors to have them built from open-source JDK's. > This creates an a problem for *alternative* tool chains. For example, > after some fumbling around I can fake out our Eclipse package, which has > explicit dependencies on the Sun JDK, and run it on classpath, jamvm, > and cacao. Ian and I have been working on this recently. Our intent to to expand the RUN_DEPENDS in java.port.mk to allow alternative tool chains to be used to run applications. > To the best of my knowledge there is not way way to tell the ports > system "make sure a 1.4.2 JVM is installed." (This would be a > categorical dependency, rather than a dependency on a specific version > or series of versions.) > > The good news is that our JDK built binaries are perfectly compatible > with these alternative tools. (The FUD about compatibility issues is > just that.) There is no reason why I should have to hack a package to > use its native binaries and jars. > > Tool chain dependencies are the most significant issue in my opinion. In addition to expanding RUN_DEPENDS we also working on unifying java application startup scripts so that things just work for the alternative jvms too. I'm not sure if these things will make it for 4.0 yet. -Kurt
Re: Java ports: source vs. binary?
> Some preliminary discussion at the last hackathon produced the > opinion that even Java ports should be built from source by all > means. That is the root of this debate - "built from source by all means". We don't have this now in the ports tree so please don't selectively apply this rule to java ports. There is a point at which building from source does not make sense or the benefits don't outweigh the effort to make it happen. While it has been pointed out that java ports can be built on fast architectures and shared with the slower archs. The realities of how java applications are built are being ignored. It is common practice for a project to include components in full from other projects by way of incorporating jar files. Let's expore this phenomena with just Eclipse and Tomcat. We build Eclipse from source, but a closer look at what you get from the source distfile reveals that a full copy of tomcat binaries (jar files) is included along with ant, junit, lucene and more. The included versions of tomcat and junit don't match our ports of these. Should we take the time to hack out these included components and force them to be built from source or to use our ports versions? What exactly do we gain by that and who is going to the work and maintain it? Continuing the analysis with Tomcat we will see that things are sightly different here but a PITA for other reasons. Let's ignore for a moment any included jar files and take a look at the release notes for the three versions of Tomcat we have. There you will find an assortment of common dependencies with different version requirements. So shall we make multiple ports of these dependencies just so that we can produce the same byte-code all over again? Oh yea and who is volunteering do all that useless busy work? I think it is time that we accept reality of the common practices of how java software is made and adapt to it. Trying to force the common practices of how C applications are built on java applications does not make sense. A few years ago I took it upon myself to get java running on OpenBSD as well as other operating systems. Now that that goal is comming to fution must I and other java oriented developers jump through flaming hoops just to port java applications to OpenBSD? -Kurt
Re: Java ports: source vs. binary?
Christian Weisgerber wrote: Some preliminary discussion at the last hackathon produced the opinion that even Java ports should be built from source by all means. However, that discussion didn't include any of our porters who are interested in Java... [My apologies of this appears twice; I posted it earlier today but it didn't show up.] I have four comments below. My perspective is from an OpenBSD porter of alternative tool chains, (JamVM, Cacao, and classpath), and as a user of J2EE frameworks for web applications. These comments are supported the following notions: 1) .jar files are a kind of distfile; 2) source availability is an important debugging aid; 3) when we trust the jar's creator we don't have to recompile; 4) a pre-built openbsd package should be worthy of being trusted; 5) source includes javadoc; 6) reducing or merging .jars raises upstream support problems; 7) recompile or not, packages with jars and libs should be usable by all tool chains; SUMMARY === -- OpenBSD should create a "verified repository" for trusted builds of pure Java frameworks. (They work on all platforms with identical bytecodes.); -- a port produces .jars, native libs, and javadoc, from source, usable by all tool chains; *BUT* for the specific framework/application, not the pure Java dependencies, which are obtained from the verified repository. -- an OpenBSD package bundles .jars and libs for all tool chains; -- a Java specific tool such as Maven may be a candidate for repository support. Fred [The discussion is intended to be self-contained for those that are not familiar with common practices in the Java community. These are my views.] PURE JAVA PORTS === A port of a pure java framework is trivial. I just copy the .jar or unzip the .zip or .tar.gz; If I need the source for debugging I get the source; and the Javadocs. If I can't do this I don't choose that framework technology and I move on to something else. This is the way the Java community works. There are often several alternatives. I get to decide this because I'm the system architect (for apps in my org.) Copy deployment works for all pure Java frameworks on all platforms supporting Java. The jar is a distfile that I don't have to compile if I am willing to trust it's creator. But it sure would be nice to know that it matches a checksum. The interesting frameworks have many dependencies. Binary distributions of those frameworks (e.g. Spring and Hibernate) may include *dozens* of other .jars that they depend on, each of which may have much more source code than the original framework. So those embedded jars are their own distfile, in addition to the source. Where did those embedded .jars come from? Clearly we are trusting the upstream developer's released package. I think a port of Spring should include checking of the distfiles of the component jars. Recompiling source is one way to accomplish this; but another is the ports "makesum" target. The difference: do I want to trust the porter? or am I going to insist to see it built with my own eyes? JAR FILE DUPLICATION & MULTIPLE VERSIONS Second, there is is a lot of jar-file duplication in real projects. Xerces can occur several times in a real web app (just to pick one favorite) depending on other frameworks being used. (You'll get three copies from Struts, Spring, and Hibernate alone.) They will all be incarnated in the same JVM. And they would not necessarily be the same version level. Individual projects are unsynchronized and older versions are likely to be around for a long time. This is reality. Now in OpenBSD we like to rationalize the libraries in our ports and eliminate duplicates, update versions where it makes sense, etc., because this improves security and fits the doctrine that OpenBSD is a complete system, not a kernel+gnu. Rationalizing the .jars across several Java frameworks with dozens of jars is a lot of work. Developers often just don't do it and instead burn the resources at execution time. The upstream developer of any particular framework just does not care. Maybe they should. Furthermore, rationalizing .jars may not change the security profile of a Java application one bit. The bigger problem is that if you try to merge versions you are likely to introduce bugs, or certainly have created a combination of jars that would be declared unsupportable by the upstream developer. How an application relates to the jars it uses by version seems to me to be much less controlled in the Java world than OpenBSD. You can call this sloppy engineering or whatever. But I don't see what value will be added by OpenBSD ports recompiling everything or merging libraries. Validating the checksums on .jars is valuable. WERE DO THE JARS GO? So you compiled all the jars. Where do you put them? It is not always clear. Do they go in / usr /
Re: Java ports: source vs. binary?
Deanna Phillips <[EMAIL PROTECTED]> wrote: > You seem to want to dismiss this (and me) because it's > inconvenient. If you choose to dismiss the users who care about > open source, what users will you be left with? Users who care about open source aren't hurt by ports of commercial software. They will be left with everyone who wants to use the OS, like they already are. Adam
Re: Java ports: source vs. binary?
Nikolay Sturm <[EMAIL PROTECTED]> writes: > * Deanna Phillips [2006-07-21]: >> One of the greatest things for me, as a user, has been that I >> can completely trust the decisions made about what does and >> does not go into this OS. Even ports. Take that away and >> what do you have? > > Those decisions don't exist. We add ports for all sorts of > crappy software. Submit a port for an interesting software > full of non-obvious bugs and it will be committed (assuming a > committer notices the submission). There are some conspicuously missing ports. I think (hope) you know this. > Just look at nagios and it's related ports, total crap but > there's nothing better out there, I needed the port, so I did > it. This simplifies life for people in a similar situation > (having to use nagios), while it certainly is no statement of > encouragement to use it. As I said in another followup, I'm not talking about the quality of ports. I'm talking about free vs non-free software, which is what this thread was about in the first place. Thinking ahead about a -trend- that could be counterproductive if followed. Not the odd port that's an exceptional case. You seem to want to dismiss this (and me) because it's inconvenient. If you choose to dismiss the users who care about open source, what users will you be left with? > I think I made my point clear, so please answer in private > mail if you want to discuss anything. No thanks.
Re: Java ports: source vs. binary?
* Deanna Phillips [2006-07-21]: > > I completely disagree. The question boils down to what our > > ports tree is supposed to be. You want it to be a packaging > > system for open source software. I want it to be a packaging > > system for any software, even closed source commercial > > software. > > To what end? Take over the desktop? Total World Domination? Installing all software in a sensible way, conforming to OpenBSD policy. On the way deal with the worst braindamage software developers come up with. > One of the greatest things for me, as a user, has been that I can > completely trust the decisions made about what does and does not go > into this OS. Even ports. Take that away and what do you have? Those decisions don't exist. We add ports for all sorts of crappy software. Submit a port for an interesting software full of non-obvious bugs and it will be committed (assuming a committer notices the submission). Just look at nagios and it's related ports, total crap but there's nothing better out there, I needed the port, so I did it. This simplifies life for people in a similar situation (having to use nagios), while it certainly is no statement of encouragement to use it. I'd do this for any software I use where some kind of distfile is downloadable (that's where I draw my personal line, but we even have a port of Maple, where you would need a cd to install it). I think I made my point clear, so please answer in private mail if you want to discuss anything. Nikolay -- OpenPGP: 0x2036A3A7 - 64E4 7D77 F5C0 EA47 A901 51EF 6E54 6E4F 2036 A3A7 "It's all part of my Can't-Do approach to life." Wally
Re: Java ports: source vs. binary?
Hello, Mr. suck my balls. Pleased to meet you. Adam <[EMAIL PROTECTED]> writes: > Deanna Phillips <[EMAIL PROTECTED]> wrote: > >> One of the greatest things for me, as a user, has been that I >> can completely trust the decisions made about what does and >> does not go into this OS. Even ports. > > What you trust and what other people trust are not the same. > There is no way for them to ensure that everyone can trust > them about what goes in ports and what doesn't. Just to clear this up -- I'm not talking about security here. I'm talking about goals.
Re: Java ports: source vs. binary?
Deanna Phillips <[EMAIL PROTECTED]> wrote: > One of the greatest things for me, as a user, has been that I > can completely trust the decisions made about what does and > does not go into this OS. Even ports. What you trust and what other people trust are not the same. There is no way for them to ensure that everyone can trust them about what goes in ports and what doesn't. > Take that away and what do you have? It doesn't exist to be taken away, what Nikolay describes is what is already in place right now. Adam
Re: Java ports: source vs. binary?
Nikolay Sturm <[EMAIL PROTECTED]> writes: > I completely disagree. The question boils down to what our > ports tree is supposed to be. You want it to be a packaging > system for open source software. I want it to be a packaging > system for any software, even closed source commercial > software. To what end? Take over the desktop? Total World Domination? One of the greatest things for me, as a user, has been that I can completely trust the decisions made about what does and does not go into this OS. Even ports. Take that away and what do you have?
Re: Java ports: source vs. binary?
Dnia 20-07-2006 o godz. 22:07 Theo de Raadt napisał(a): > > It's pervert to have a STOP BLOB release theme and then importing > > exactly BLOBS in the ports tree. There is absolutely no need to do so, > > nothing suffers from going throught the source, besides, maybe these > > ports are a little bit harder to do. > > Please do not misuse the term BLOB like this. This issue is not what > we use the word for. We call it a blob when there is no source, and > when there is no license to use it. These are not real blobs. > > Without constantly asking for source, vendors will provide only binary (who needs source if all use 'convinient' binaries) and all will be undocumented and blobby. :-[ You will see... -- Pozdrawienia/Regards Tomasz Zielinski Cesaria Evora we Wrocławiu! Królowa World Music zaśpiewa 6 sierpnia podczas Wrocław Summer Guitar & Folk Festival. http://klik.wp.pl/?adr=http%3A%2F%2Fadv.reklama.wp.pl%2Fas%2Fcesaria2.html&sid=820
Re: Java ports: source vs. binary?
> I was - as naddy pointed out - very outspoken on this issue during c2k6. > And I still am. > > I am against ports that download pieces of code that do not have their > source form in /usr/ports/distfiles. > > I want at least to be able to see what the program does by inspecting > the sources. And I do not want to rely on any third party webpage to > get at the sources. I want it in distfiles. And, I want to be able to > apply patches, if needed. > > We do not hurt our users by requiring the availability of source code. > It's only the package builders that need the jdk installed. Our users > are told to use packages anywas. They just install the package. > > But the port should never, ever, rely on precompiled stuff. Not for > Java, not for Python, not for Ruby. And not for anything else. > > Source code should always goto distfiles, be unpackaged, compiled on the > porters machine and be transferred to a package. > > What's next? Binary only software with NOT_FOR_ARCHES set so it runs > only the arch the binary is for? > > It's pervert to have a STOP BLOB release theme and then importing > exactly BLOBS in the ports tree. There is absolutely no need to do so, > nothing suffers from going throught the source, besides, maybe these > ports are a little bit harder to do. > > You are 100% correct, I support You. All binary-only software should be limited as much as possible. Sources can be analyzed, patched, changed, audited, ported to other platform. Why, why use binary blob if we have a choice. Today we have choice. In the future, if nobody wants sources, vendors will provide only obscure binaries. Do we all want it? -- Pozdrawienia/Regards Tomasz Zielinski Oglądaj -> Wydarzenia.wp.pl -> Wiesz, co najważniejsze! Najbardziej aktualne materiały filmowe - Kliknij: http://klik.wp.pl/?adr=www.wydarzenia.wp.pl&sid=821
Re: Java ports: source vs. binary?
On Fri, Jul 21, 2006 at 01:35:32PM +0200, Marc Espie wrote: > Somebody explain to me how slower platforms are a hastle for java-based > ports. The same reasons cross-compiling isn't supported: * supporting cross-builds is extra work * you still need to build on the slower platform periodically to prevent bit-rot. Regards, Andrew Dalgleish
Re: Java ports: source vs. binary?
Isn't the jvm code supposed to be platform-independent ? No, that's a misunderstanding. The JVM is the platform-dependant runtime. It is Java class files (aka byte code) that are platform independent. Just like: A sh or perl script may be portable; the OpenBSD that it runs on is arch-dependant. There is completely no reason to build platform-independent stuff on slow arches, you can just copy over the packages from the fast arch... So, like in ports/books, any Java port that generates only class files (no native code) should have PKG_ARCH= * to allow this sort of copying. as far as I know, the big issue with java and slow architectures is that java isn't as portable as it is supposed to be, and need arch-dependent shit to work... Flame troll, ignored :-) (and for Nikolay, we still have a few binary-only packages. There are reasons for these, such as commercial apps/semi-commercial apps like opera, but in general, if source is available, then source should be used) Redhat and freebsd-libs. The source is available.
Re: Java ports: source vs. binary?
What's next? Binary only software with NOT_FOR_ARCHES set so it runs only the arch the binary is for? Well, yes. redhat-base and freebsd-libs set for only i386. It's pervert to have a STOP BLOB release theme and then importing exactly BLOBS in the ports tree. There is absolutely no need to do so, nothing suffers from going throught the source, besides, maybe these ports are a little bit harder to do. As Theo has pointed out, these are not blobs. > I am against ports that download pieces of code that do not have their > source form in /usr/ports/distfiles. Good. Where is your patch to emulators/redhat-base to make it cross-compile all of Linux userland (including gcc)? Nobody except you will suffer from this besides, maybe, this port being "a little harder to do".
Re: Java ports: source vs. binary?
Marc Balmer wrote:
I was - as naddy pointed out - very outspoken on this issue during c2k6.
And I still am.
I am against ports that download pieces of code that do not have their
source form in /usr/ports/distfiles.
I want at least to be able to see what the program does by inspecting
the sources. And I do not want to rely on any third party webpage to
get at the sources. I want it in distfiles. And, I want to be able to
apply patches, if needed.
We do not hurt our users by requiring the availability of source code.
It's only the package builders that need the jdk installed. Our users
are told to use packages anywas. They just install the package.
But the port should never, ever, rely on precompiled stuff. Not for
Java, not for Python, not for Ruby. And not for anything else.
I disagree here.
The ports system should be a way to help getting 3rd party software to
run on OpenBSD, with no restrictions other than the one imposed by the
3rd party provider themselves.
The port system could have more clear flags than the PERMIT_* ones to
mark the packages for which no source code is available, and those for
which source code is available, but were built from pre-built binaries.
This would allow people to choose their application according to their
own policy.
SOURCE_CODE_AVAILABLE=Distfile||No
BUILT_FROM_SOURCE_CODE=Yes|No
Distfile: litteral meaning that ${DISTFILE} is the source code
: Url of the source code if ${DISTFILE} is not source
And put that in the PLIST.
> What's next? Binary only software with NOT_FOR_ARCHES set so it runs
> only the arch the binary is for?
So you're suggesting rm -rf
/sys/compat/{bsdos,freebsd,ibcs2,osf1,sunos,ultrix,hpux,linux,netbsd}
too ?
--
Matthieu Herrb
Re: Java ports: source vs. binary?
On Fri, Jul 21, 2006 at 06:45:52PM +1000, Damien Miller wrote: > On Fri, 21 Jul 2006, Bernd Schoeller wrote: > > > On Thu, Jul 20, 2006 at 06:12:54PM +0200, Christian Weisgerber wrote: > > > We need some sort of policy how to deal with software written in > > > Java. We have a number of ports that are basically just wrappers > > > that install pre-compiled Java byte code. > > > > Please have the java source in the ports tree, and the .class/.jar > > stuff in the packages. Without going into the political details, this > > is just what I would expect as a user. > > This is all very nice, but someone has to do the work. Kurt, who does > do the work, says that maintaining source-based ports is too much hassle > for him on slower platforms. So if people really want this approach then > you need to be prepared to help out more than happens now. Somebody explain to me how slower platforms are a hastle for java-based ports. Isn't the jvm code supposed to be platform-independent ? There is completely no reason to build platform-independent stuff on slow arches, you can just copy over the packages from the fast arch... as far as I know, the big issue with java and slow architectures is that java isn't as portable as it is supposed to be, and need arch-dependent shit to work... There are probably mixed ports out there, for which it would be feasible to have `mixed' builds. Just grab the bytecode from the fast arch. In my opinion, there are *two* distinct points, and you need to make sure you understand both. - open software. We have to encourage build from source, so that source stays available. This is about feasible for every java port, and at least the fast architectures should take that route, all the time. I fully agree with Theo on that one: when in doubt, take the source. (and for Nikolay, we still have a few binary-only packages. There are reasons for these, such as commercial apps/semi-commercial apps like opera, but in general, if source is available, then source should be used) - practicality. Building anything on fast machines is reasonable, in the developer's world. End-users shouldn't care. End-users can use packages. If they whine about compilation times, that means they use the ports. If they use the ports, they want to tinker and tweak, hence they need the source. (we're not gentoo, `tweaking' is a bit more than hacking at obscure compiler options to try to get one extra percent of performance out of a program at the cost of extra bugs). When building everything isn't reasonable, package stuff so that slow machines can use the fast machines packages. End of story...
Re: Java ports: source vs. binary?
On Fri, 21 Jul 2006, Bernd Schoeller wrote: > On Thu, Jul 20, 2006 at 06:12:54PM +0200, Christian Weisgerber wrote: > > We need some sort of policy how to deal with software written in > > Java. We have a number of ports that are basically just wrappers > > that install pre-compiled Java byte code. > > Please have the java source in the ports tree, and the .class/.jar > stuff in the packages. Without going into the political details, this > is just what I would expect as a user. This is all very nice, but someone has to do the work. Kurt, who does do the work, says that maintaining source-based ports is too much hassle for him on slower platforms. So if people really want this approach then you need to be prepared to help out more than happens now. -d
Re: Java ports: source vs. binary?
On Thu, Jul 20, 2006 at 06:12:54PM +0200, Christian Weisgerber wrote: > We need some sort of policy how to deal with software written in > Java. We have a number of ports that are basically just wrappers > that install pre-compiled Java byte code. Please have the java source in the ports tree, and the .class/.jar stuff in the packages. Without going into the political details, this is just what I would expect as a user. With ports, I want to be able to control the compilation process, apply patches and browse the source. Packages are for "just running", ports give me much more. If I do not want to waste compilation time, I use packages (which is where OpenBSD and Debian are much more sane than Gentoo or FreeBSD, where compiling from source seems to be a matter of lifestyle). The only consequence of "Java binaries run on all platforms" is that the same package files can be shared between the different hardware architectures in "packages". That is more or less the same with packages of scripting languages or documentation. Bernd
Re: Java ports: source vs. binary?
On Thu, 20 Jul 2006 19:22:13 +0200, "Martin Schröder" <[EMAIL PROTECTED]> said: > 2006/7/20, Theo de Raadt <[EMAIL PROTECTED]>: > > > Leave installing the source as an option to the user, and install > > > binaries as a default. > > > > And in 5 years noone will make source available. > > Better: Install the source where possible (and warn if there is no > source) but don't install from source as a default (and make this > choice configurable), and use binaries where provided, thus speeding > installation. > > I doubt that the typical eclipse user wants to compile eclipse from > source. :-) There's a 'reason' the community is called 'Open Source'. This movement towards binaries defeats the entire purpose. The typical user shouldn't have to build from source, anyway. That is not what is being proposed, as I understand it. The typical user should never have to build from source. That's why there are those package things so many people like to ignore. -- Eric Furman [EMAIL PROTECTED]
Re: Java ports: source vs. binary?
On Thu, Jul 20, 2006 at 06:50:11PM +0200, Martin Schröder wrote: > Installing from source also needs a jdk, while installing binaries > only needs the runtime environment. This only means that those who build from ports also need a complete jdk. Ordinary users can (and should) just install packages where appropriate. Ciao, Kili
Re: Java ports: source vs. binary?
It seems like there are a number of questions to answer in this discussion. I can find (or think of myself) at least the following ones: 1. Shall each port be able to fetch the source for all its bytecode? (given that this is both legal and technically possible to implement) 2. Shall each port that can fetch such source also fetch it by default? 3. Shall each port that can fetch such source also be able to build the bytecode from that source? 4. Shall each port that can build the bytecode from the source also build it by default? 5. Can a port be accepted even if it violates any of the above decisions? (And if so: Under which circumstances? And which warnings or other special measures shall be used to handle that port?) 6. Can a port be accepted when it is illegal or techically impossible for the port to fetch the source for all its bytecode? (And if so: Under which circumstances? And which warnings or other special measures shall be used to handle that port?) 7. Shall support be added in the ports system for building a file for just a single target and then reuse the result on a number of target platforms? And as several people has noted, these questions does not only relate to java bytecode, but also to all kinds of generated files, including documents and run-time libraries. /Johan ps. I know how I would answer most of the questions, but I am not a port maintainer. (To be honest, I rarely have the time to even install anything else than packages.) So I try to avoid giving brilliant solutions that everyone else knows would fail ;-) ds.
Re: Java ports: source vs. binary?
* Marc Balmer [2006-07-20]: > I am against ports that download pieces of code that do not have their > source form in /usr/ports/distfiles. I completely disagree. The question boils down to what our ports tree is supposed to be. You want it to be a packaging system for open source software. I want it to be a packaging system for any software, even closed source commercial software. > What's next? Binary only software with NOT_FOR_ARCHES set so it runs > only the arch the binary is for? We have such ports since ages. Nikolay
Re: Java ports: source vs. binary?
> It's pervert to have a STOP BLOB release theme and then importing > exactly BLOBS in the ports tree. There is absolutely no need to do so, > nothing suffers from going throught the source, besides, maybe these > ports are a little bit harder to do. Please do not misuse the term BLOB like this. This issue is not what we use the word for. We call it a blob when there is no source, and when there is no license to use it. These are not real blobs.
Re: Java ports: source vs. binary?
I was - as naddy pointed out - very outspoken on this issue during c2k6. And I still am. I am against ports that download pieces of code that do not have their source form in /usr/ports/distfiles. I want at least to be able to see what the program does by inspecting the sources. And I do not want to rely on any third party webpage to get at the sources. I want it in distfiles. And, I want to be able to apply patches, if needed. We do not hurt our users by requiring the availability of source code. It's only the package builders that need the jdk installed. Our users are told to use packages anywas. They just install the package. But the port should never, ever, rely on precompiled stuff. Not for Java, not for Python, not for Ruby. And not for anything else. Source code should always goto distfiles, be unpackaged, compiled on the porters machine and be transferred to a package. What's next? Binary only software with NOT_FOR_ARCHES set so it runs only the arch the binary is for? It's pervert to have a STOP BLOB release theme and then importing exactly BLOBS in the ports tree. There is absolutely no need to do so, nothing suffers from going throught the source, besides, maybe these ports are a little bit harder to do.
Re: Java ports: source vs. binary?
On Thursday 20 July 2006 12:12 pm, you wrote: > We need some sort of policy how to deal with software written in > Java. We have a number of ports that are basically just wrappers > that install pre-compiled Java byte code. Additional ports in this > style have been proposed. Actual Java source may or may not be > available, but it is certainly not used by the ports in question. All the ports I maintain have src available. IFAIK all java ports have src available. There have been various replies to this thread that assume otherwise but please take a moment to look first. Please stop continuing this FUD. > Some people--Marc Balmer has been very outspoken--dislike this > approach, because we are just wrapping other people's binaries. Lets be clear here, these are java byte-code binaries that run inside the jvm. No native executable is installed from a third party. > Instead, ports should fetch the source and newly compile the code. > The counter argument from the Java people is that Java byte code > is machine-independent, compiling it afresh will just produce the > very same binaries, adding build time for no gain. An additional > complication is that passing around binary archives seems well-accepted > in the Java scene, posing problems of obtaining the actual source > code and exploding dependency requirements. It is not a matter of accessing the source code. It is a matter of trust, build time and unnecessary complications to porting new java programs to OpenBSD. Take tomcat for example. Apache has been distributing this for many many years. There are no native componets to tomcat. Pure java byte-code. The same byte-code runs on Windows, Linux, Solaris, *BSD, etc. etc. The whole world trusts the byte code released by Apache. Why should we not? If we convert tomcat/v4 to build from src for example, there will be a *significant* increase in package build time on arm and sparc especially. Currenly devel/jdk/1.3 doesn't even need to be built by bulk package builders to package tomcat/v4. Building tomcat from src in ports will add unnecessary complications of exploding the number of dependencies. For what gain? > How are we going to deal with this? > > Some preliminary discussion at the last hackathon produced the > opinion that even Java ports should be built from source by all > means. However, that discussion didn't include any of our porters > who are interested in Java... The source requirement may render > various ports impossible or impracticably difficult. We'll need to > decide whether we put our foot down here. For the last few years, java on OpenBSD has been mostly a one man show. I was hoping by now there would be more involvement from other folks. If we take what I consider to be an unreasonable stance on this we might as well delete all the java ports because who is going to do the work and maintain it all? I certainly cant. -Kurt
Re: Java ports: source vs. binary?
> P.S. Good time to re-read Ken's paper Reflections on Trusting > Trust (online at http://www.acm.org/classics/sep95/), before you decide > where to put your foot down. Wow, that is totally off-topic. The discussion is about how we can use some of our clout to encourage source availability in the long term (versus giving no pressure at all) while not hurting our users by not giving them what they need. Right from the start, the discussion is about where to draw the line, in a field of grey, instead of choosing black versus white position. THAT is the problem we always face. A way to balance them. When I run software that is in ports, I have absolutely no doubt that there are security risks, and quite frankly, I don't care. If you care, you must work to fix it, and in reverse too -- if you don't work to fix it, then you must not care enough to fix it. And that is NOT what this is about. When binaries have holes, or risk factors due to distribution problems, well DUH, so does source code. ANY SOURCE CODE has just as much risk. Which of you even bloody read what you download. It is the standard balony of "If it is signed it must be risk-free" or "If it is signed at least it has fewer risks". It is still balony. This discussion is not about security. It is about trying to encourage suppliers of code to stick to the open source way of doing things. If we accept binary files all the time, how are we sending the right message? But if we only accept source, we are hurting our users. THERE ARE MANY MIDDLE GROUNDS where we don't hurt our users too much, yet we send a message to people who don't supply source. Even with "blobs", we have middle grounds, because while some of them are huge traps, some of them (like firmwares) just require redistribution rights, but too few people actually study the situation well enough and then (stupidly) they choose either black or white positions. Again, this discussion has ZERO TO DO WITH SECURITY.
Re: Java ports: source vs. binary?
Wow, that is totally off-topic. Yes, it was. It's about open source.
Re: Java ports: source vs. binary?
Christian Weisgerber wrote: We need some sort of policy how to deal with software written in Java. We have a number of ports that are basically just wrappers that install pre-compiled Java byte code. Additional ports in this style have been proposed. Actual Java source may or may not be available, but it is certainly not used by the ports in question. The source code is available for all of the programs AFAIK. If anybody thinks I have put anything into the ports tree for which source is not available, let him quote chapter and verse. The source for the JDK - which you need to compile any of the others - does require manual downloading because of a click-wrap license. That is not the issue. The source for all of them is available. So is the source for redhat-* and freebsd-libs, but we've been shipping those as binary blobs for years. There are ~30 RPMs in redhat/base alone, including X libraries. You're going to recompile all of that on every bulk build of ports? Some people--Marc Balmer has been very outspoken--dislike this approach, because we are just wrapping other people's binaries. Instead, ports should fetch the source and newly compile the code. The counter argument from the Java people is that Java byte code is machine-independent, compiling it afresh will just produce the very same binaries, adding build time for no gain. That's the same argument as using PDFs in /books. First, if the book is only available in PDF, should we refuse to install it because it might have malicious code that would trigger a buffer overrun in xpdf or one of the libraries? Second, if a book is available in PDF but also in LaTex, should we insist that somebody install all of LaTex just to build it? And further, should the ports build machines have to do this on every platform (even vax :-)), or, should there be a means in the ports mechanism to say that "the package that this builds works on any platform" - this would apply to building PDF and building Java, among others. And it also applies to the linux-libs and freebsd-libs, etc. If you are going to be consistent and say "build everything", you are going to be setting up cross-compilation and building all the linux libraries and the freebsd libraries every time you do a full ports build. An additional complication is that passing around binary archives seems well-accepted in the Java scene, It is, when they are from a recognized site like Apache or Eclipse or Sun. Just as people use packages from an OpenBSD mirror. And we use linux packages from redhat, PDF packages from Sun, O'Reilly, and others. posing problems of obtaining the actual source code and exploding dependency requirements. The dependency requirements are the real issue. The source for the IDEs is huge; source for one of the other ports (glassfish, IIRC, a J2EE server) is larger than that of OpenBSD kernel+user together. How are we going to deal with this? Some preliminary discussion at the last hackathon produced the opinion that even Java ports should be built from source by all means. However, that discussion didn't include any of our porters who are interested in Java... The source requirement may render various ports impossible or impracticably difficult. We'll need to decide whether we put our foot down here. Put a SECURITY file in pkg/ for anything that hasn't been completely built by us, whether it's Java, PDF, a port that installs linux or freebsd libraries, or even a port that installs a wireless card blob. And, make sure there is a link where to get the source for those who have time to spend downloading and building every prerequisite. Ian P.S. Good time to re-read Ken's paper Reflections on Trusting Trust (online at http://www.acm.org/classics/sep95/), before you decide where to put your foot down.
Re: Java ports: source vs. binary?
+-- | On Thursday, Jul 20, 2006 at 06:12:54PM +0200, Christian Weisgerber wrote: | | To: ports@openbsd.org | From: Christian Weisgerber <[EMAIL PROTECTED]> | Date: Thu, 20 Jul 2006 18:12:54 +0200 | Subject: Java ports: source vs. binary? | | We need some sort of policy how to deal with software written in | Java. We have a number of ports that are basically just wrappers | that install pre-compiled Java byte code. Additional ports in this | style have been proposed. Actual Java source may or may not be | available, but it is certainly not used by the ports in question. | | Some people--Marc Balmer has been very outspoken--dislike this | approach, because we are just wrapping other people's binaries. | Instead, ports should fetch the source and newly compile the code. | The counter argument from the Java people is that Java byte code | is machine-independent, compiling it afresh will just produce the | very same binaries, adding build time for no gain. An additional | complication is that passing around binary archives seems well-accepted | in the Java scene, posing problems of obtaining the actual source | code and exploding dependency requirements. | | How are we going to deal with this? | | Some preliminary discussion at the last hackathon produced the | opinion that even Java ports should be built from source by all | means. However, that discussion didn't include any of our porters | who are interested in Java... The source requirement may render | various ports impossible or impracticably difficult. We'll need to | decide whether we put our foot down here. | | -- | Christian "naddy" Weisgerber [EMAIL PROTECTED] It would be nice to make people only have ports with source. Seems to go against the whole "blob" thing to accept bytecode compiled Java. If there was a policy and I had a vote, I'd say make source only a requirement. Other than standing on the open source front for an argument, is there a practical argument that relates to reality? How many people have time audit the Java code, seems it is hard enough to just get a port reviewed and committed as it is. Java bytecode is easier to reverse than machine code, yet it is still blobby. I think the option should be available for source auditing in the interest of security, should it ever come needed or someone feels like auditing Java ports for some reason and enforced by policy for ports. Very few Java ports ports for things would not get added/submitted if I had to guess. Java community is different from the UNIX c community, yet it shouldn't matter in the interest of security right? You should be able to audit and make changes to the source if you wanted to, for whatever reason. People will use whatever Java software they want anyways, just having OpenBSD package it for them is handy. I personally would like to have the option available to me to view the source, should I ever want to some day. Having OpenBSD package it up for and integrated into the package management system is just a nice thing to have. IMO, ports should follow the same goals as OpenBSD listed on goals.html, and closed source Java bytecode doesn't seem to fit in there, especially with the latest anti-blob campaign. -Chris -- a programmer with free time, sometimes.
Re: Java ports: source vs. binary?
On Thu, Jul 20, 2006 at 06:50:11PM +0200, Martin Schr?der wrote: > Installing from source also needs a jdk, while installing binaries > only needs the runtime environment. > > Leave installing the source as an option to the user, and install > binaries as a default. RUN_DEPENDS = jre BUILD_DEPENDS = jdk Ports builders have always had more dependancies than package installers. -- David Terrell [EMAIL PROTECTED] ((meatspace)) http://meat.net/
Re: Java ports: source vs. binary?
2006/7/20, Theo de Raadt <[EMAIL PROTECTED]>: > Leave installing the source as an option to the user, and install > binaries as a default. And in 5 years noone will make source available. Better: Install the source where possible (and warn if there is no source) but don't install from source as a default (and make this choice configurable), and use binaries where provided, thus speeding installation. I doubt that the typical eclipse user wants to compile eclipse from source. :-) Best Martin
Re: Java ports: source vs. binary?
> Leave installing the source as an option to the user, and install > binaries as a default. And in 5 years noone will make source available.
Re: Java ports: source vs. binary?
* Christian Weisgerber [2006-07-20]: > We need some sort of policy how to deal with software written in Java. > We have a number of ports that are basically just wrappers that > install pre-compiled Java byte code. Additional ports in this style > have been proposed. Actual Java source may or may not be available, > but it is certainly not used by the ports in question. We have a somewhat similar situation with ports based on ruby gems. There it is simply not feasible to do otherwise or at least I haven't figured out how to build from the source archives. Wrt Java, there's the problem, that the Java source code is not readily available, thus we would need manual intervention to build source based packages. I say, do what the user community is used to. If they prefer distributing source code, great. If they prefer distributing byte code, oh well. Nikolay
Re: Java ports: source vs. binary?
Installing from source also needs a jdk, while installing binaries only needs the runtime environment. Leave installing the source as an option to the user, and install binaries as a default. Best Martin
Re: Java ports: source vs. binary?
> How about using the source if it's available and using the binary > when it's not? Right. And in 5 years, how much source will you have? > Don't forget, having the source also means being able to patch it > as well. Duh. The point of Christian's mail was is that we all understand how these things work. If you keep accepting non-source components, eventually noone ever makes source available. Using source if it is available is not enough of a policy to ENCOURAGE SOURCE AVAILABILITY.
Re: Java ports: source vs. binary?
On Thu, Jul 20, 2006 at 06:12:54PM +0200, Christian Weisgerber wrote: > We need some sort of policy how to deal with software written in > Java. We have a number of ports that are basically just wrappers > that install pre-compiled Java byte code. Additional ports in this > style have been proposed. Actual Java source may or may not be > available, but it is certainly not used by the ports in question. > > Some people--Marc Balmer has been very outspoken--dislike this > approach, because we are just wrapping other people's binaries. > Instead, ports should fetch the source and newly compile the code. > The counter argument from the Java people is that Java byte code > is machine-independent, compiling it afresh will just produce the > very same binaries, adding build time for no gain. An additional > complication is that passing around binary archives seems well-accepted > in the Java scene, posing problems of obtaining the actual source > code and exploding dependency requirements. > > How are we going to deal with this? > > Some preliminary discussion at the last hackathon produced the > opinion that even Java ports should be built from source by all > means. However, that discussion didn't include any of our porters > who are interested in Java... The source requirement may render > various ports impossible or impracticably difficult. We'll need to > decide whether we put our foot down here. How about using the source if it's available and using the binary when it's not? Don't forget, having the source also means being able to patch it as well. -Ray-
Re: Java ports: source vs. binary?
On Jul 20, 2006, at 10:12 AM, Christian Weisgerber wrote: Some preliminary discussion at the last hackathon produced the opinion that even Java ports should be built from source by all means. However, that discussion didn't include any of our porters who are interested in Java... The source requirement may render various ports impossible or impracticably difficult. When I use Java software, I like very much to build it myself, from NetBeans to JEdit. On one hand, that's an extremely nerdly position to take. On the other hand, the OJ community is by definition extremely nerdly! But all in all, it seems to me that on OpenBSD there is an expectation that pretty much anything that one would wish to install should come with source if only for non-repudiation purposes. I.e., if there is an exploit hidden in the software, you want the exploit to have arrived on your machine from source you can examine to determine the culprit. --- Jack J. Woehr Director of Development Absolute Performance, Inc. [EMAIL PROTECTED] 303-443-7000 ext. 527
