Re: Moving Apache Harmony to the Attic
Parts of the Harmony classlibs live on in Mika too, and maybe ultimately parts of DRLVM too. I recognise the inevitability of this move, but I can't bring myself to vote for it. Harmony proved a lot of things, including the applicability of the Apache model to a huge project involving engineers from several companies and a specification you would pay not to have to implement. Congratulations (and heaps of thanks) to all those involved. All the best Chris Gray
Re: Rebooting the Harmony project
[David Beberman] We are a commercial JVM as well, Aicas. Not sure that is relevant for this discussion though. Kind of hard to tell what would be relevant to a discussion of this type. Maybe OP meant to say, whilst there are at least a dozen implementations of the Java VM there is currently only one of the Parrot VM? At the risk of being on-topic, I personally would be very sorry to see DRLVM completely orphaned - there's a bunch of good stuff in there. Chris
Re: [classlib][nio] JDK 7 / WatchService support
2011/2/27 Ray Chen clrayc...@gmail.com Hi Jimmy, Thank you for the information! I have a question here, jpathwatch is in GPL license, can harmony use that? I am not an legal expert, just curious. Good question Ray, and I think it's OK if the owner hold the copyright and re-license it. ACQ and ICLA should be the legal insurance. GPL + classpath exception FWIW. (Personally I am baffled by the classpath exception; I haven't a clue what this library or independent modules translate to in Java terminology). If Uwe is sole author then he can just add the option to use the code under ASL. Chris
Re: strong type cast needed in java. (overriding enhancement.)
Hi Prashanth, I believe that all these kinds of features i.e. base/sub class access confusion, friend functions were all purposely avoided in JAVA to make life simple for the programmers and to get rid of crashes that might possibly occur on supporting the same. I agree. The simple syntax of Java already creaking under the weight of enhancements such as generics, autoboxing, and varargs already interacting to create subtle gotchas - and there's more to come in Project Coin et al. There's a trade-off between intended convenience and behind-the-scenes complexity, and I think Java may already be heading for the wrong side of that trade-off. Anyway, Apache Harmony is a Java runtime not a compiler, and the features which are being talked about here primarily concern the compiler. So this would be a matter for an experimental fork of ecj or gcj ... best regards Chris
Re: roadmap suggestion
Hi Paul, Exactly.I suggested additionally that if it avoids patent problems, alternative bytecode could be used instead (like Dalvik's, since it's already widespread) as long as that doesn't block performant support of non-Java languages on the VM - while reusing as much of Harmony's existing guts as possible. Well, at the risk of repeating myself: I suspect that switching to a different bytecode (1) would raise an obstacle to migrating languages which currently target the JVM to the proposed neo-Harmony and (2) would not necessarily solve any patent problems. 1) Although application developers are flocking to the Android APIs, I don't see language developers flocking to Dalvik - on the contrary, the apparent restriction to Dalvik and the operating environment it expects are often perceived as disadvantages of Android. Does anybody have a different impression? 2) Of the seven patents which Oracle claims Google is infringing: http://en.swpat.org/wiki/Oracle_v._Google_(2010,_USA)#The_patents I don't see one which would be circumvented by using a different bytecode, but SFAICS it is possible to build a JVM without infringing any of them. The same goes for the patents which are explicitly mentioned in the JVM spec - they relate to optimisations and not to the bytecode itself. Did I miss something? [...] - In the short term, could Harmony be released as it is, certified only for its ability to run the runtimes of a set of VM languages like Python and Ruby? Its class library implementation is surely solid enough. The bytecode output of the compiler could be changed later to match an updated VM if needed, perhaps with a temporary intermediate step, after the corresponding work on the language runtimes. By the compiler do you mean the compiler of the non-Java language which has the Harmony VM as target? Or do you mean that ecj should emit Dalvik (or some other) bytecode? Best regards Chris
Re: AW: Harmony could be a world beater with the right strategy, not rot in the attic!
Hi Paul, I don't want to pour cold water on your idea but I would like to point out some counter-arguments. I'm thinking of a VM that would reuse Java's syntax (as Dalvik does), but would not be tied to being 'Java' in the TCK/OpenJDK sense. It would reuse de facto standard bytecodes (i.e. Dalvik's) for 'run anywhere'. Porting of apps and frameworks might be done by simple recompilation or by bytecode conversion as with Java-to-Android for phone apps. Using a different bytecode imposes an extra step in the build chain, and perhaps more importantly raises an obstacle to bytecode engineering, especially at runtime. I see just these aspects becoming increasingly important in beyond-Java languages, particularly in the areas of aspect-oriented programming and highly dynamic or interactive languages. A lot of languages have already been developed with Java bytecode as a target, or have been ported to Java bytecode: http://en.wikipedia.org/wiki/List_of_JVM_languages I am not aware of such a list for Dalvik or other alternative virtual machines. Maybe if we build it they will come, but we are not going to be immediately able to plug into Scala and Fantom and all that stuff. Dalvik has been designed to run any program that can be written in Java (and even then there are some rough edges, ask the guys that ported an OSGi framework to Dalvik); I'm not sure that it's the case that any legitimate Java bytecode (i.e. anything which would survive verification) can be automatically translated to Dalvik. Maybe it can, but I would like to see a formal proof. The sword of Damocles of patents would not then hang over the VM, This sword hangs over every line of code that has been written in the last 30+ years and every line of code that will be written in the foreseeable future, and Dalvik is no exception. When Dalvik was first announced I didn't buy the argument that Google would be able to circumvent all patent issues just by using a different VM and not calling it Java. I still don't. Would this be feasible for Harmony as an alternative to a JVM release or mouldering exile in the attic? Technically it is feasible, but I don't really see what problem it solves. For me it would be more interesting to come up with a minimal, stable set of Java class libraries which could be relied upon by JVM-based implementations of non-Java languages. Scala for example exposes underlying Java libraries which is fine except that the underlying JRE could be anywhere in the range 5 = n 9 if you see what I mean. I think that the developers of these languages would be much happier with the Apache way of life than with the JCP. Of course I'm interested in other VM instruction sets too, the JVM is very far from being the Last Word. But then I'm not sure that Java or JVM compatibility is a good starting point, it would be better to look at a more modern language design and see what kind of VM support it really needs. Kind regards Chris Gray
Re: [general] Harmony future roadmap
In message 4ccfe67a.8040...@gmail.com, Tim Ellison writes: On 31/Oct/2010 20:23, chris.g...@k-embedded-java.com wrote: Tim Ellison wrote: I think all options and opinions are open for discussion. We could simply continue with the current goal and encourage the Apache Board to seek the JCK license by whatever means is posible, we could modify our goal and plan to release an uncertified Java SE runtime, or more radically change Harmony's goal away from Java SE. Previously the situation was that Harmony didn't *want* to pass the JCK, Harmony *had* to pass the JCK in order to be able to make a release. Well, not really. We have always said that Harmony's goal was for a compatible and compliant implementation of Java SE. Passing the JCK has a number of advantages, including the ability to demonstrate to our users that Harmony is 'real' Java. Indeed. Passing the TCK is key to our goal to be compatible and compliant. We *need* to pass the TCK to meet the goal of being compliant, but we also *want* to pass the TCK as it is a good measure (arguably the best measure) of our being compatible. Passing the TCK was also presented as being the key to SUN's intellectual property rights that are essential to practice this specification. Of course a company that can refuse access to the TCK can also find an excuse not to grant those rights anyway. Passing the TCK is also important to gaining credibility with some potential users. But that was all based on agreements and contracts which have now been unequivocally abrogated, so I guess it doesn't matter much any more. In which case I think Harmony should turn its back on the JCK and instead seek to be as compatible as possible by whatever means is posible. I've certainly hear others making that argument too -- that is to say, continue with the goal of of full Java compatible runtime, and accept that Harmony won't be certified. Of course, there are other assurances and rights that come with passing the JCK, and so it would be interesting to hear the importance that contributors and consumers of Harmony place on that. To me as a potential contributor it matters not one hoot. Consumers may have other reactions, of which more anon. A year ago several people were suggesting that we might take a stand and release Harmony. To take one random example: http://www.jroller.com/niclas/entry/oracle_s_java_community_commitment Even ignoring changes since that time, I think this misses the important point that no matter how safe we, the ASF, think we are there is no point making releases that no one (with more to lose?) would risk using. Among users (and potential users) of alternative runtimes that I have talked to there is a very broad range of viewpoints. Some indeed (those with the most active License Review Boards) do take the line that they would not want to use anything which carries the tiniest risk of legal complications. (Of course this risk can never be eliminated entirely, witness the Wang/Kodak debacle as well as SCO). Others are surprisingly sanguine about the whole affair, apparently regarding the current Oracle/Google kerfuffle as a kind of celebrity scandal diversion rather than something that might affect them directly. Yes, on the whole larger companies tend to be more cautious than small ones in this respect but I still know some major players who are using unlicensed JREs in some product or other. So my problem with taking anything but the continue with the current goal and encourage the Apache Board to seek the JCK license by whatever means is posible option is that I believe that the other options I can imagine are rather short on potential users. That said, from a purely technical perspective some of the other options are rather appealing so I'd be happy to be convinced my non-technical conclusions are wrong. Precisely - from a technical point of view one would prefer to think that this was all just some silly tantrum on the part of Java's new owners/custodians and once it is all over Java will be freer than ever. But I have to admit that it may turn out otherwise and then some of us may have to accept that our efforts have been in vain. For the time being I prefer to keep the flame burning. Best regards Chris Gray
Re: [general] Harmony future roadmap
Hi Tim, I think all options and opinions are open for discussion. We could simply continue with the current goal and encourage the Apache Board to seek the JCK license by whatever means is posible, we could modify our goal and plan to release an uncertified Java SE runtime, or more radically change Harmony's goal away from Java SE. Previously the situation was that Harmony didn't *want* to pass the JCK, Harmony *had* to pass the JCK in order to be able to make a release. But that was all based on agreements and contracts which have now been unequivocally abrogated, so I guess it doesn't matter much any more. In which case I think Harmony should turn its back on the JCK and instead seek to be as compatible as possible by whatever means is posible. The only question is whether Harmony should try to track the whole of Java 7 and 8 or try to stick with a core and leave the rest to other (possibly Apache) projects. Modularising Java in this way would already be a radical departure from the way Sun/Oracle have been handling the development of Java, and Harmony clearly has something to contribute here. I have an inkling that such a modularised Java would be popular in enterprise computing - but as you know my personal interest is in the embedded kind. In the past Harmony (or at least Geir) was reluctant to open up a second front on the embedded side, but now there seems to be nothing to be gained by such restraint ... I'd like to make an analogy with Darwin. (Yes, I'm a closet FreeBSD guy.) Darwin is basically the FreeBSD userland that Apple distributes as part of OS X, without the apple-specific GUI. (Apple also use a different OS kernel.) So for FreeBSD substitute Harmony, and for Apple substitute another commercially successful company (which uses a different VM), and what do we have? I think we have a very interesting modular approach to Java for mobile and embedded. [1] e.g. http://s.apache.org/xI BTW this link seems to be broken. Best regards Chris
Re: [classlib][x-net] Creating a provider based on OpenSSL
Hi Oliver, - Set a single global set of native RNG callback functions that then figures out (somehow) who made the original call into the OpenSSL natives and uses the appropriate SecureRandom implementation. This would be tricky, but might be possible. I imagine we could look up the stack for the last Java (i.e. non-native) frame and could get the SecureRandom reference from that class. I went for something like this option in the end using ThreadLocals to get back my SSLParameters class when we're in the RNG functions (thanks to Cath Hope for this idea!). I'm not completely satisfied with the solution, but it works and is fairly simple. In the case where no SecureRandom implementation is provided the code falls back to the OpenSSL default RNG functions. The only problem I see with this is that it assumes that all calls to a particular SSL engine instance will be made from a single thread. That may not be the case if for example the SSL connection is established by a main or startup thread and then messages will be sent on that connection by an EventListener which is run by a quite different thread. That would lead to extremely puzzling bugs 8-0. From that point of view I would prefer an implementation which simply ignored the user-supplied SecureRandom (and clearly documented this behaviour). I also have to say that if this is the biggest unsolved problem you still have, I'm pretty impressed. Best regards Chris Gray
Re: [classlib][x-net] Creating a provider based on OpenSSL
Hi Oliver, I'm currently investigating the possibility of implementing a JSSE provider wrapping OpenSSL. This would be a great idea, but don't underestimate he task. IIRC there is at least one abandoned project on SourceForge to do something similar. I also know of at least one proprietary (company-internal) project which does something of the sort, and there are probably many others - all implementing just as much of JSSE as the company in question needs, and no more. At first sight there seems to be little commonality between the two APIs, so actually implementing the whole of JSSE looks like a daunting task. Don't let me put you off though! Best regards Chris Gray
[tools] Problem building apache-harmony-5.0-src-r946978
Hi all, Trying to build from source (apache-harmony-5.0-src-r946978) results in the following: -really-download: [echo] Fetching depends/jars/xerces_2.9.1/xerces.zip [get] Getting: http://www.apache.org/dist/xml/xerces-j/Xerces-J-bin.2.9.1.zip [get] To: /home/chris/apache-harmony-5.0-src-r946978/classlib/depends/jars/xerces_2.9.1/xerces.zip [get] Error opening connection java.io.FileNotFoundException: http://www.apache.org/dist/xerces/j/Xerces-J-bin.2.9.1.zip [get] Error opening connection java.io.FileNotFoundException: http://www.apache.org/dist/xerces/j/Xerces-J-bin.2.9.1.zip [get] Error opening connection java.io.FileNotFoundException: http://www.apache.org/dist/xerces/j/Xerces-J-bin.2.9.1.zip [get] Can't get http://www.apache.org/dist/xml/xerces-j/Xerces-J-bin.2.9.1.zip to /home/chris/apache-harmony-5.0-src-r946978/classlib/depends/jars/xerces_2.9.1/xerces.zip I tried changing ´xerces-j´ to ´xerces/j´ in classlib/make/depends.properties and common_resources/make/depends.properties, but then it turns out that http://www.apache.org/dist/xerces/j/ doesn´t contain 2.9.1 versions, only 2.10.0. Am I doing something wrong? Platform: Ubuntu 9.04, procedure: download and unpack apache-harmony-5.0-src-r946978-snapshot.tar.gz, cd into apache-harmony-5.0-src-r946978/, type ´ant´, read friendly error message and type ´ant fetch-depends´. Regards Chris
Re: Idiomatic Java: inverted conditions
On Tue, 27 Oct 2009 14:21:14 +, Tim Ellison t.p.elli...@gmail.com
wrote:
On 27/Oct/2009 13:35, Xiao-Feng Li wrote:
On Tue, Oct 27, 2009 at 6:51 PM, Tim Ellison t.p.elli...@gmail.com
wrote:
On 26/Oct/2009 21:57, Jesse Wilson wrote:
Continuing along with a theme, there's another C/C++ism in our Java
code
that frustrates me. Our Java code frequently inverts conditions from
their
natural language form.
I'm sure we all have our own horror stories. The ones that make me
cringe are structured like this,
public void foo(Object bar) {
if (bar != null) {
...
some long method, typically /too/ long
...
return result;
}
throw IllegalArgumentException();
}
Grrr.
Tim
LOL. The code examples from Jesse, Alexey and Tim are all interesting.
When I saw those styles, I usually just assumed the authors must have
their strong justifications for that, since sometimes I saw the code
from some seasoned programmers and they refused to give an
explanation. :) I might guess the original intention of the authors
is to help the (unwise) compiler to produce expected efficient code.
For example, with the code above, the author may expect the compiler
be silly and instruct it to produce fall-through code for the
most-frequently-taken branch. Well, with modern
microprocessor/compiler, this kind of code is (almost) no longer
needed.
I agree Xiao-Feng, and that is a poor reason to write code in this style
anyway. I would even suggest that coding for the compiler and coding
for the next person who has to read the code are usually complimentary,
such as marking fields as final, and using the most restrictive possible
method scope, etc. Of course, these details are usually swamped by
higher-level algorithm choice anyway.
Some of these are indeed C-isms - e.g. if (constant == variable) because
if you mistakenly typed if (constant = variable) the C compiler would
give you an error whereas with if (variable = constant) it would not. (In
fact gcc will issue a warning for this pattern, so even this is moot). But
others are based on well-intentioned rules such as conditionals should be
positive, i.e. if the conition is true this should indicate that everything
is OK, or show the normal path of excution first and the abnormal paths
afterward. These rules sound fine until you try to put them into practice.
As for the one return point at the end rule, I personally don't see a
big difference between
if (bar == null) return null;
and
if (bar == null) throw new IllegalArgumentException();
So for me early reaturns are OK in this kind of context, especially it
there is enough whitespace around that they are not likely to be
overlooked.
Greets
Chris Gray
--
Chris Gray /k/ Embedded Java Solutions
Re: Probs with checking certificates from JarInputStream
Hi Tim, Chris Gray wrote: snip Note that this code doesn't even call closeEntry() on the meta-files it encounters, and yet it still works on RI 1.4/1.6. On harmony the *.DSA and *.SF files are simply skipped, without any entries being added to the metaEntries map. Adding an else clause in which meta-files are first exhaustively read([BII) and then closeEntry() is called seems to help a lot. A shortcoming of Harmony for sure. I think the answer is to treat all meta-files the way we now treat the manifest, i.e. provided they are presented in the right order we read them all in during the JarInputFile constructor and extract whatever data will later be needed by the JarVerifier. Right order means something like: meta- files precede all non-meta files, manifest precedes all except possible the META-INF/ directory itself. This implies that we also suck in the META- INF/INDEX.LST file and omy[deity] the META-INF/services directory if present, unless we can insist that these come after the signature data. Instead of mEntry we would have a whole queue of entries buffered and waiting for getNextEntry() to fish them out. Eek. Is this based on any reading of the spec, or experimentation, or just your hunch about the order in which these things should be considered? Apart from the fact that there are numerous places we should be looking, what happens when there are differences of opinion between them, etc.? Good question. The jar file specification is informal to say the least, but there does seem to be some kind of consensus that for instance the META-INF/MANIFEST.MF entry needs to come pretty darn early in the archive if it's to be of any use - the existing Harmony code only allows META-INF/ to precede it. It also seems reasonable to insist that the .SF and .DSA entries have already been read in when the file to be signed is encountered (because the signature becomes available once the file has been read fully). OTOH I'm not so sure that all the META-INF/* entries need to be together - for all I know it would be legal to put META-INF/INDEX.LST at the very end, for example. I'm not a fan of long-running constructors (imagine if the jar file were being read in from a slow server over a chunked HTTP stream - been there, done that), but in this case it's probably justified; arguably the JarInputStream object is not fully constructed until the signatures have been processed. Yeah, though there are any number of misbehaving data streams that could cause such delays in the JarInputStream construction today that I think we are already in that game. In fact it seems to help a lot with the jar signing if I simply add code to JarInputStream.getNextEntry which calls read([BII) exhaustively if the current file has the meta nature, before invoking super.getNextEntry. We want verifier.metaEntry to be called, and this is a side-effect of read([BII); so if the caller didn't read the file then we do it, throw away the result and keep the side-effect. :-0 Do I need to do contributor paperwork before submitting small patches like this? Regards Chris Chris Gray /k/ Embedded Java Solutions _ Scarlet says goodbye to download limits! ADSL20 NO LIMIT, only 29,95 Go to www.scarlet.be for more info!
Re: Probs with checking certificates from JarInputStream
On Friday 08 May 2009 17:33:45 Alexei Fedotov wrote:
Hello Chris,
It seems SignatureTest2.java keeps vanishing. Could it be that some
paranoid anti-virus scanner removed the attachment?
Pooh, probably Scarlet's honky webmail. Blocking source files and letting
through executable jars, that's what I call smart ... this time both attached
and inline, using good ol' SMTP. Scroll down further and I will ramble on
about a possible solution.
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.util.Enumeration;
import java.util.jar.*;
class SignatureTest2 {
static public void main(String[] args) {
try {
String filename = args.length == 0 ? SignedTestBundle_1.0.0.jar :
args[0];
System.out.println(Opening file: + filename);
System.out.println();
FileInputStream fis = new FileInputStream(filename);
BufferedInputStream bis = new BufferedInputStream(fis, 8192);
JarInputStream jis = new JarInputStream(bis);
Manifest m = jis.getManifest();
JarEntry je = jis.getNextJarEntry();
while (je != null) {
System.out.println(JarEntry: + je);
if (!je.isDirectory()) {
String name = je.getName();
if (!name.startsWith(META-INF/)) {
jis.closeEntry();
java.security.cert.Certificate[] certs = je.getCertificates();
if (certs == null) {
System.out.println(No certificates);
}
else {
System.out.println(Certificates:);
for (int i = 0; i certs.length; ++i)
System.out.println(certs[i].toString());
}
System.out.println();
}
}
je = jis.getNextJarEntry();
}
}
catch (Throwable t) {
t.printStackTrace();
}
}
}
Note that this code doesn't even call closeEntry() on the meta-files it
encounters, and yet it still works on RI 1.4/1.6. On harmony the *.DSA and
*.SF files are simply skipped, without any entries being added to the
metaEntries map. Adding an else clause in which meta-files are first
exhaustively read([BII) and then closeEntry() is called seems to help a lot.
I think the answer is to treat all meta-files the way we now treat the
manifest, i.e. provided they are presented in the right order we read them
all in during the JarInputFile constructor and extract whatever data will
later be needed by the JarVerifier. Right order means something like: meta-
files precede all non-meta files, manifest precedes all except possible the
META-INF/ directory itself. This implies that we also suck in the META-
INF/INDEX.LST file and omy[deity] the META-INF/services directory if present,
unless we can insist that these come after the signature data. Instead of
mEntry we would have a whole queue of entries buffered and waiting for
getNextEntry() to fish them out.
I'm not a fan of long-running constructors (imagine if the jar file were being
read in from a slow server over a chunked HTTP stream - been there, done
that), but in this case it's probably justified; arguably the JarInputStream
object is not fully constructed until the signatures have been processed.
What a mess. What a format. What an API.
Cheers
Chris
--
Chris Gray/k/ Embedded Java Solutions BE0809.435.306
Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/
chris.g...@kiffer.be +32 3 216 0369
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.util.Enumeration;
import java.util.jar.*;
class SignatureTest2 {
static public void main(String[] args) {
try {
String filename = args.length == 0 ? SignedTestBundle_1.0.0.jar : args[0];
System.out.println(Opening file: + filename);
System.out.println();
FileInputStream fis = new FileInputStream(filename);
BufferedInputStream bis = new BufferedInputStream(fis, 8192);
JarInputStream jis = new JarInputStream(bis);
Manifest m = jis.getManifest();
JarEntry je = jis.getNextJarEntry();
while (je != null) {
System.out.println(JarEntry: + je);
if (!je.isDirectory()) {
String name = je.getName();
if (!name.startsWith(META-INF/)) {
jis.closeEntry();
java.security.cert.Certificate[] certs = je.getCertificates();
if (certs == null) {
System.out.println(No certificates);
}
else {
System.out.println(Certificates:);
for (int i = 0; i certs.length; ++i)
System.out.println(certs[i].toString());
}
System.out.println();
}
}
je = jis.getNextJarEntry();
}
}
catch (Throwable t) {
t.printStackTrace();
}
}
}
Probs with checking certificates from JarInputStream
Hi, We've started adopting Harmony classes for the core APIs of the Mika VM, currently on a somewhat ad-hoc basis (taking a small group of classes at a time and manually downgrading them to 1.4 level) Mostly we're very happy with the results but occasionally we run into problems which I'd like to report here. Attached SignatureTest2 class is based on an application which wants to check that a jar file was signed by a recognised principal. For this it creates a JarInputStream and walks through it using getNextEntry(), calling getCertificates() on each entry. This code works on RI 1.4 and 1.6, but fails (getCertificates() returns null) on Harmony, Classpath (at least on the one from Debian 4) and RI 1.5 - the last is the subject of a bug report no. 6284489. So Harmony is in good company here, but the customer would prefer to be in the more select company of those who get it right. :-) It looks to me as if the problem arises because method readCertificates() of JarVerifier is called when only the manifest has been read, so no signatures are found in the metaEntries map. However my efforts to fix this so far have not met with success, maybe someone who knows the code better will have an idea how to fix it? Best regards Chris Gray /k/ Embedded Java Solutions _ Scarlet says goodbye to download limits! ADSL20 NO LIMIT, only 29,95 Go to www.scarlet.be for more info! TestBundle_1.0.0.jar Description: application/java-archive
Re: [general] Reduced footprint class library
Hi Tim, I can see this working in Mika, in fact I'm tempted to implement it already. :-) How are you getting on -- did you try it?! I'm cracking on with the 'non-dynamic' model of removing the optional data in NIO_CHAR. No, I didn't have time yet, due to customer meetings etc.. However at one of those meetings we had a request to provide java.nio on Mika, so it's definitely time to look at the following: You can now go into the NIO_CHAR/make/excludesfile and specify converters to drop from the Harmony provider. That allows you to ditch some esoteric charsets and save footprint, and in many cases the ICU provider has an equivalent anyway. But isn't the ICU provider itself rather large? The native charsets look a bit verbose, with their sparsely-populated arrays and boilerplate code repeated in different files. I think we might want to tweak this a bit ... for java.lang.Character we built some quite compact structures, for example re-using identical table rows (e.g. all 0x00). But we do this at build time based on the selected charsets, and it would be harder if charsets could be added dynamically (so say nothing of unloading them!). In fact it's pretty hard to be dynamic *and* compact *and* start up fast. :-( I suppose the Sun-defined provider mechanism only works for 100%-Java charsets? Regards Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [general] Reduced footprint class library
On Monday 29 September 2008 17:46, Tim Ellison wrote: Chris Gray wrote: You can now go into the NIO_CHAR/make/excludesfile and specify converters to drop from the Harmony provider. That allows you to ditch some esoteric charsets and save footprint, and in many cases the ICU provider has an equivalent anyway. But isn't the ICU provider itself rather large? Yes. We can move on to look at reducing that too, but first order was to reduce the duplication we are responsible for. On closer inspection it appears that there are both Java-only and native versions of quite a range of popular charsets, without having to import ICU - so I'm pretty happy from that point of view. It also looks to be fairly easy to code up a new one (e.g. for JIS) ad-hoc if needed. The natives are there to support heuristic optimization of encoding/decoding direct buffers. While it makes sense on a machine with lots of resources (e.g. a server) it doesn't for constrained devices which is why they are now optional. See above. I suppose the Sun-defined provider mechanism only works for 100%-Java charsets? Not sure I understand the question. Sorry, unclear expression of my muddled thinking. Of course the classes loaded in by the framework can also load native libraries, so long as the class loader in question knows how to find them. I was thinking the libraries would have to be on the standard library path, but it ain't so. You can write a provider with non-Java code too (like the native code we have got), but you then hook into the provider mechanism through the Java APIs. Of course, you can also have 'built-in' providers that are not added via the regular framework APIs, which we use for example in bootstrapping the class libraries. Yes of course. Regards Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [general] Reduced footprint class library
On Thursday 18 September 2008 17:31, Tim Ellison wrote: Right. In fact, as you know Harmony has NIO (for IO channels) and NIO_CHAR (for character converters). We went through an exercise way back to split the monolith into different functional areas, and defined the metadata to describe the inter-dependencies. While we try to minimize the coupling some of it is forced upon us by the SE APIs. Yep, and this makes Harmony a much more promising starting-point for such an exercise than the alternative(s). In actual use cases the most popular single import seems to be java.beans (which does actually correspond to a Sun-defined profile, namely CDC/PP). The biggest import was in order to run Ant on the target for purposes of auto-deployment - not a route I would recommend, but that was the customer's choice. A good example. Harmony's BEANS module depends upon AWT through a spurious public interface parameter (and the fact that BEANS carries around the persistence delegates). For people that don't want AWT and SWING in their runtime they need to break the SE API in Beans. What to do about cases like this? Do we have classes in the build path which are not in the released artefact? Real ones or just-to-make-it-compile stubs? And afterwards do we leave those methods in the class file (so if someone uses the method it will break at runtime), or do we bytecode-engineer them out and hence break binary compatibility? Regards, Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [general] Reduced footprint class library
On Friday 19 September 2008 14:05, Tim Ellison wrote: Chris Gray wrote: On Thursday 18 September 2008 17:31, Tim Ellison wrote: A good example. Harmony's BEANS module depends upon AWT through a spurious public interface parameter (and the fact that BEANS carries around the persistence delegates). For people that don't want AWT and SWING in their runtime they need to break the SE API in Beans. What to do about cases like this? Do we have classes in the build path which are not in the released artefact? Real ones or just-to-make-it-compile stubs? And afterwards do we leave those methods in the class file (so if someone uses the method it will break at runtime), or do we bytecode-engineer them out and hence break binary compatibility? I think it depends on the world you envisage when somebody steps outside the scope of the released artifact: If you assume a 'closed' world, then I suggest you don't want to have these 'mines' left in the runtime, because when somebody calls such a method you will get a guaranteed runtime exception (ClassNotFound or whatever). So, as you say, in a closed world the choice is either to pre-process the source code to remove the methods, or post process the class files to hide/remove them. Either way you are not binary compatible with SE. For me the normal use case is that the world is 'closed' wrt java.* and javax.*. This is the case when for example an OSGi bundle is compiled against a given Execution Environment - specifiying the EE is like a shorthand for a manifest entry Import-Package: java.lang,java.util,java.net,java.io, The developer of a module needs to know something about the environment it will run on; if java.awt and/or javax.swing are not supported then this makes a big difference. In this case it is best to remove the unsupported members from the class, so that an attempt to use them can be detected at compile time instead of at run time. If you assume a 'dynamic, connected' world, then you would leave the methods in and download/expand the runtime when somebody uses a missing method. Provided the application stays within the boundaries of the released artifact all is well, but it would mean that the first call to BeanInfo.getIcon() takes a really long time while the runtime goes off to fetch AWT ! Well, the first call to AWT normally takes some time to execute anyway ... for me java.awt is not a particularly likely candidate for this kind of lazy loading, but anyway I can see that it could make sense to ship the VM with some minimal set of packages and to download others on demand. In that case maybe the bootstrap or extension classpath could contain placeholder jar files which contain only and INDEX.LST and a pointer to where the real jar can be found. The index is needed so that the bootstrap classloader knows which classes are to be loaded from the referenced location; the first time one of these has to be loaded, the bootstrap classloader goes and fetches the real file and overwrites the placeholder with it. From that point on loading proceeds as normal ... In this case you would certainly leave the unresolvables in the class file, they will be resolvable when the time comes. I can see this working in Mika, in fact I'm tempted to implement it already. :-) Regards Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [general]Mika and Harmony
Hi Tim, Sounds good to me. Let's see if anyone else is interested. Should we start a separate thread with a more obvious title? You spurred me into committing the patch I was sat on to reduce the footprint of the NIO character converters, by making the native implementations optional. You can now delete the 1.5Mb hyniochar library and we'll fall back to doing the conversions in Java code. I'm sure there are more examples like this where we can have the best of both worlds for 'big' server and 'small' embedded sets of libraries. In fact JavaME CDC/FP doesn't include java.nio, with the result that I regularly have to patch code that is using java.nio just to do simple byte-char conversions. But in other respects this is a good example - for a large class of applications (those which make only incidental use of char conversion) the java-only library is perfectly adequate and saves 1.5 MB (handy if your platform only has 8 MB of ROM!). Another area I would like to look at is Locale - last time I looked Harmony's Calendar was relying on a 6 MB library to handle Locale-related stuff, it could be interesting to have a lightweight alternative. Best regards Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [general] Reduced footprint class library
On Thursday 18 September 2008 15:10, Tim Ellison wrote: In another thread [1], Chris wrote: Should we start a separate thread with a more obvious title? Done, tweak the subject as you see fit. Ta. [...] In fact JavaME CDC/FP doesn't include java.nio, [...] True, as you are adhering to the ME specs. But if you are prepared to branch out and define an alternative reduced footprint library profile you might also choose to include NIO functionality, and 1.5 syntax support, etc. You may have some insights based on the Mika class libraries with packages taken from Harmony use cases you have already seen. Java.nio is something of an Awful Example in this context, because it contains such disparate elements. One application might really need the charset stuff but have no use whatever for the java.net-revisited parts, another might be i/o intensive and need direct buffering and select()-style functionality above all. IOW the granularity of the footprint reduction process will probably not map neatly onto package boundaries. In actual use cases the most popular single import seems to be java.beans (which does actually correspond to a Sun-defined profile, namely CDC/PP). The biggest import was in order to run Ant on the target for purposes of auto-deployment - not a route I would recommend, but that was the customer's choice. See also my response to Alex Blewitt regarding char converters and locales. [1] http://markmail.org/message/x3ydeos244pqpcws Regrads, Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [general] Reduced footprint class library
Hi Alex, On Thursday 18 September 2008 15:27, Alex Blewitt wrote: Converters are needed as an on demand basis - is there any reason the dynamic loading of a charset could be augmented with the ability to download additional converters if not found locally? This might not make sense for non-connected devices but for desktops could allow a minimal set to be distributed and then others to be acquired on demand. I think you need to split this into two parts - first a flexible (but lightweight) mechanism for resolving charsets or locale data on the fly, and then secondly downloading from a remote server as one way of retrieving the data. Where the data is determined by the Locale the mechanism could be based on java.util.ResourceBundle; then it becomes a question of how to control the search mechanism so that in some cases it will include a remote repository. Regards Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [general]Mika and Harmony
Hi Tim, [...] I think there is a role for Mika if we go down the road of producing a reduced class library configuration out of Harmony, or one more suitably targeted to limited memory environments. I assume that a combination of Mika + full Harmony class libraries is not going to be so interesting. No, I don't think that Mika + full Harmony is particularly useful; however people have been known to extend the Mika class libraries with packages taken from Harmony, to support some library they want to use on the target. Mika is a BSD-licensed VM designed for the JavaME CDC space; the class libraries it uses are currently mostly its own but moving towards using Harmony. Currently that movement is somewhat glacial (we have to dumb-down Harmony to 1.4 compliance, which is boring work), but it will get a boost once we start work on supporting Java5 features. Yep. I can't imagine a great rush to backport the Harmony class libraries to 1.4 syntax compliance, so I think it would be an important goal to bring Mika up to 1.5 bytecode support. Well if anybody else does have the need to make 1.4 backports of Harmony classes I would like to meet them ... In fact the Mika VM doesn't need such a lot of work to support 1.5 bytecode, it's really just a matter of providing access to signatures and annotations. (We already support ldc class.) Of course to use these features we would need new class libraries, and that's where Harmony comes in. What do people think - does Mika have a role to play in Harmony or should we just be downstream users of the Harmony classlibs? I think it would work fine either way. We can work closely within the Apache project -- but if you are working fine as a downstream consumer then the question to ask would be 'what would you expect to change by coming into the fold?' If there are other people interested in creating smaller profiles from Harmony then we could coöperate on defining those profiles and making them easy to build. We'd particularly like to define a profile which is to Java5 as CDC/FP is to 1.4, if you see what I mean. And we would be bringing a VM which can run those classes on a lot of embedded platforms - ARM, MIPS, PowerPC, maybe soon SH4 too. Best regards Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
[general]Mika and Harmony
Hi All, I'd like to re-raise the question of whether the Mika VM (http:// en.wikipedia.org/wiki/Mika_VM, http://www.k-embedded-java.com/mika/trac/) has a role to play within the Harmony project. I discussed this previously with Geir Magnusson, but that was before he moved on ... Mika is a BSD-licensed VM designed for the JavaME CDC space; the class libraries it uses are currently mostly its own but moving towards using Harmony. Currently that movement is somewhat glacial (we have to dumb-down Harmony to 1.4 compliance, which is boring work), but it will get a boost once we start work on supporting Java5 features. What do people think - does Mika have a role to play in Harmony or should we just be downstream users of the Harmony classlibs? Best rgards Chris -- Chris Gray/k/ Embedded Java Solutions BE0503765045 Embedded Mobile Java, OSGihttp://www.k-embedded-java.com/ [EMAIL PROTECTED] +32 3 216 0369 Skype: k.embedded.chris
Re: [classlib][drlvm][swing] test TEST-javax.swing.text.DefaultStyledDocumentRTest leads to VM crash
On Thursday 23 November 2006 07:02, Vladimir Ivanov wrote: No, it is time for lunch :) Eh? I only just had breakfast ...
