Re: [drlvm] the big soup of VM properties (HARMONY-1626)
Hello All, I finished implementation and refactoring described in Harmony-1925. In H-1925 You can find some details and patch file (whole_properties_patch.zip). Any comments are welcome. I observe the following further improvements (Items for other JIRA-s?): - In function read_properties (vm/vmcore/src/init/properties.cpp): Property value has limited length (MAX_PROP_LINE=5120). This function should be rewritten. - We should establish order with vm.boot.library.path, vm.boot.class.path properties. Either we should add api2vm interface which should bring INTERNAL_PROPERTIES to java code or we should introduce other JAVA_PROPERTIES with functionality of properties described above and rewrite existing kernel classes. Thanks Dmitry 2006/10/20, Dmitry Yershov [EMAIL PROTECTED]: Hi All, I finished implementation of new properties module for DRL VM. Also, HARMONY-1925 was created to track the progress. You can find some details and patch file here. Any comments are welcome Thanks Dmitry
Re: [drlvm] the big soup of VM properties (HARMONY-1626)
2006/10/17, Mikhail Fursov [EMAIL PROTECTED]: Is there any progress in this area? Just an interest, because I think that properties refactoring is rather important task. Yes, I'm working on it. I'll provide first patch soon. This patch will introduce new properties module according my proposal and yours advises. Then I'll provide second patch which does full re factoring in VM properties area. + Does anybody knows the answer on the following question: All JIT settings are VM properties. User can change the behaviour by redefining them. Does it affect TCK certification? I don't know the answer, but current implementation does not store JIT settings into VM properties. It enables these settings through function: (*jit)-next_command_line_argument(-Xjit, arg); (see parse_arguments.cpp) Dmitry On 10/12/06, Gregory Shimansky [EMAIL PROTECTED] wrote: On Wednesday 11 October 2006 09:41 Mikhail Fursov wrote: +1 to Eugueni and Alexey and -1 to use String type in the intercomponent interface. + AFAIK the String type is VM internal type only. Ok you've convinced me. +1 for const char * On 10/11/06, Alexey Varlamov [EMAIL PROTECTED] wrote: 2006/10/11, Evgueni Brevnov [EMAIL PROTECTED]: Gregory, My 2cents: 1cent) I think we should not integrate property module so tight (by using StringPool) with VM internals. Let it be independent enough. 2cent) I also don't think we should pollute StringPool any further. Instead I would like to see if we can get it smaller say by splitting into two pools whatever... Agreed. Besides properties are used in VM almost solely during init, no performance gain here. So -1 to using Strings for properties. Evgueni On 10/11/06, Gregory Shimansky [EMAIL PROTECTED] wrote: On Tuesday 10 October 2006 11:36 Geir Magnusson Jr. wrote: Inline Dmitry Yershov wrote: [snip] VM properties proposal == The general purpose of VM Properties subcomponent is to provide centralized access to a common properties table. A property is meant as a pair of key and value. The properties stored in VM Properties table represent configuration settings for a separate component (such as VMCore, GC, JIT etc) or for the whole system. Another use case for the properties is communication between different components. Requirements 1) The key and value are represented as string (i.e. char*). and I propose that on each operation, a copy is made, so that the caller frees the string that they got or gave. Hmm... I thought of another type. VM has a String class which represents an internal strings implementation. String pointers all refer to the same interned const char * memory location so comparing them is faster, you just compare pointers. Would it be better to have key and value be String * instead of const char *? It would save memory allocation, copying and comparing and lookup in properties hash table could be done using a pointer. I don't think it is a very performance critical place, properties aren't accessed very often, but at least it may reduce memory footprint and will cause less memory leaks when someone forgets to free a returned copy. On the other hand, String storage is global to the whole VM, so there are tons of strings kept inside of it. Lookup inside of a small hash table like properties may be much faster than lookup through all the strings that VM keeps... Hmm now I am not sure I want to suggest this way, just want it to be considered. -- Gregory Shimansky, Intel Middleware Products Division - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Gregory Shimansky, Intel Middleware Products Division - Terms
Re: [drlvm] the big soup of VM properties (HARMONY-1626)
[snip] Requirements 1) The key and value are represented as string (i.e. char*). and I propose that on each operation, a copy is made, so that the caller frees the string that they got or gave. Agree. .. [snip] 1) if this is a C++ class, lets drop the word 'property' We can not drop this word because these interfaces are exported from VM core. To my mind external interfaces should have useful names. 2) Don't hardcode 'hidden' into the API, as we may have other types down the road. Make it an argument Ok. So, the latest list of interface functions should be: void set_property(const char * key, const char * value, bool type) const char * get_property(const char * key, bool type) bool is_property_set(const char * key, bool type) void unset_property(const char * key, bool type) char * * get_properties_keys(bool type) char * * get_properties_keys_staring_with(const char * prefix, bool type) If there are no other objections i'll file a jira, start redeveloping property module, and then do source code refactoring. Dmitry - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [drlvm] the big soup of VM properties (HARMONY-1626)
[snip] Requirements 1) The key and value are represented as string (i.e. char*). and I propose that on each operation, a copy is made, so that the caller frees the string that they got or gave. Agree. .. [snip] 1) if this is a C++ class, lets drop the word 'property' We can not drop this word because these interfaces are exported from VM core. To my mind external interfaces should have useful names. 2) Don't hardcode 'hidden' into the API, as we may have other types down the road. Make it an argument Ok. So, the latest list of interface functions should be: void set_property(const char * key, const char * value, bool type) const char * get_property(const char * key, bool type) bool is_property_set(const char * key, bool type) void unset_property(const char * key, bool type) char * * get_properties_keys(bool type) char * * get_properties_keys_staring_with(const char * prefix, bool type) Why bool type? Indeed we may want more types in future. And, AFAIU VM-exported interfaces are in C. Let it be int ? Thanks Alexey. Of course it should be int. Dmitry If there are no other objections i'll file a jira, start redeveloping property module, and then do source code refactoring. Dmitry - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [drlvm] the big soup of VM properties (HARMONY-1626)
2006/10/10, Mikhail Fursov [EMAIL PROTECTED]: On 10/10/06, Alexey Varlamov [EMAIL PROTECTED] wrote: I think there is correlation between these 2 questions :). If we use const char*, we simply copy key and value when setting a property. And this is impossible with void*. Ok, this way it does works! BTW, there is alternative functionality named VMLocalStorage - so why make properties competing with VMLS? Do we really need hidden properties? +1. Want to know the answer. Seems, that we need hidden properties. Users can define any properties through command line -Dkey=value. Our VM has a set of dangerous properties (vm.dlls, vm.ee_dlls, vm.other_natives_dlls ...). We should hide their from users because redefining of their may lead to a crush. On the other hand users can define such properties and they should be visible for java applications. If You know other solution, lets discuss it. Dmitry -- Mikhail Fursov - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [drlvm] the big soup of VM properties (HARMONY-1626)
There are my thoughts about VM properties. I observe some problem with current VM properties implementation: 1) The current VM properties initialization is rather chaotic: * Some properties stored into two places: Properties class and array of VmStandardProperty structure. A certain algorithms use properties from first place. But other algorithms use properties from second place. * There are plenty of functions working with properties (on initialization stage, see create_vm function of vm_main.cpp). Number of it may be reduced (and I see some ways to do that). * There are two arrays contain predefined properties. * There is very interesting implementation of InsertSystemProperties function (see java_lang_VMExecutionEngine.cpp). Properties described in this function should be defined earlier. This function should read their from Properties table. 2) Redefining by user (through command line option –Dkey=value) some properties (i.e. vm.other_natives_dlls) crashed VM. 3) VM properties table is not thread safe. So, I propose the following: 1) Redevelop VM properties module according to attached proposal. This implementation: * May be based on apr hashtable. * Thread safe. * Introduces public (for java and VM side) and private (for VM side only) properties. It allows us to avoid VM crashes when some key properties (i.e. vm.other_natives_dlls) redefined by user. 2) Do deep source code refactoring. After that we should have clear and strong algorithms of property initialization and usage. Some redundant VM properties and arrays of predefined properties should be throw out on this stage. Dmitry 2006/10/6, Geir Magnusson Jr. [EMAIL PROTECTED]: So one of the interesting things that jumped out of me is the number of various library and boot classpath properties that we have floating around, and I'd like to 1) understand this 2) get rid of all the ones that we don't need Is there a defined set we need to support? Anyone have suggestions? geir VM properties proposal == The general purpose of VM Properties subcomponent is to provide centralized access to a common properties table. A property is meant as a pair of key and value. The properties stored in VM Properties table represent configuration settings for a separate component (such as VMCore, GC, JIT etc) or for the whole system. Another use case for the properties is communication between different components. Requirements 1) The key and value are represented as string (i.e. char*). 2) NULL values are allowed. 3) VM Properties subcomponent exports its interface as part of VM Core. 4) There are two kind of properties * Public properties are properties visible both to all VM components and to Java applications (by means of java.lang.System.getProperties() method). * Hidden properties are used for VM components communication only, not accessible from Java side. 5) There is a way to set hidden properties through non standard command line option ( -XDefinekey=value). 6) There is a way to get a list of defined properties which keys begin with a particular prefix. 7) Properties table should be thread safe. VM Properties usage scenario Components can define properties and lookup them by name. When setting the property it can be marked either public or hidden. A property should be marked as hidden if it is intended internal usage only. Public properties are supposed to be visible from Java side also. In the same time it is possible to obtain a list of all public properties to pass them to Java side. Defined Interface functions === void set_property(const char * key, const char * value) - Sets the public property. NULL values are supported. void set_hidden_property(const char * key, const char * value) - Sets the hidden property. NULL values are supported. const char * get_property(const char * key) - Returns the value of the property if it has been set by set_property function. Otherwise returns NULL. const char * get_hidden_property(const char * key) - Returns the value of the property if it has been set by set_hidden_property function. Otherwise returns NULL. bool is_property_set(const char * key, bool hidden) - Checks if the property is set. void unset_property(const char * key, bool hidden) - Unsets the property. char * * get_public_property_keys() - Returns an array of keys for all public properties set. char * * get_properties_staring_with(const char * prefix, bool hidden) - Returns an array of keys for all
Re: [classlib] porting to other platforms
At first, all assembler files should be propagated to the 64bit platforms. e.g.: classlib/modules/luni/src/main/native/thread/linux/thrspinlock.s classlib/modules/luni/src/main/native/thread/linux/thrhelp.s or classlib/modules/luni/src/main/native/thread/windows/thrspinlock.asm classlib/modules/luni/src/main/native/thread/windows/thrhelp.asm Thanks, Dmitry. 2006/7/25, Marina Goldburt [EMAIL PROTECTED]: Hi, I'm interested in the task of porting classlib to the 64bit platform (em64t/amd64). At this moment, classlib's source structure and build system doesn't support the diversity of platforms. Let's discuss what changes have to be made to support other platforms. One way is to move platform-dependent to the appropriate directory. For example, port--linux --ia32 --em64t --ipf makefile and to add platform determination and proper files selection to the makefile. The other way is to select the platform-dependent code by the preprocessor difine directives and to include the platform determination and the difines setting to the top-level build files (unfortunatly it doesn't work with asm files). Any thoughts? Thanks, Marina. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [drlvm] proposals for VM internationalization
Hello all.
Salikh Zakirov wrote:
far below are results of my experiments with Log4cxx's ResourceBundle.
(I've managed to find it in Log4cxx documentation after carefully
rereading your original post).
The good news is that it does localization (severely limited).
The prototype has following good properties
* The unlocalized message is used as the message key
The message key should be message pattern (not a message), because
some parameters may be in this message. E.g.:
Message pattern with integer parameter: %d
or
Message pattern with one parameter: {0}
* No extra entities were introduced (like non-printable message keys)
What about very long message pattern (e.g. see help message from
VM)? For these cases messageId key should be used.
* The localizable messages are marked by _() notation and can
be extracted from the source code automatically
To my mind solution is graceful for localizable messages extraction.
But should we care about this? Once, these messages should be gathered
and put into properties file.
I propose the following solution:
Modify VM's LoggerString class. The first parameter of composite
message should be message key. If it equals empty string then the
message should not be localized. E.g.:
WARN( Not localizable message with two parameters: 1 and 10)
WARN(localizable message with two parameters: %d and %d 1 10)
or
WARN(localizable message with two parameters: {0} and {1} 1 10)
The things that I have not implemented yet (to save time and make at least
something available):
* loading the system locale value
* reading the locale-specific localization file
* converting the localized messages to locale-specific encoding
What do you mean there?
* converting the unlocalized messages from source encoding (US-ASCII) to
UTF-16 (wchar_t[])
There is big question. We can use there char[] strings. Log4cxx
automatically converts char* to wchar_t*.
Also, we can use utf8 coding for wide characters.
The issues that I have encountered but haven't yet worked out a solution:
* PropertyResourceBundle.getString().c_str() returns the
pointer to the stack
location. To make it work, I had to use wcsdup(), thus introducing
an unacceptable memory leak.
I think there must be some way to get the pointer to original bundle
contents,
but haven't figured out how to achieve it.
May be that's the way:
LOG4CXX_DECODE_WCHAR(chstr, wchrstr);
LOG4CXX_ENCODE_CHAR(charstr, chstr);
charstr.c_str()
* PropertyResourceBundle expects the good property format, so the
unlocalized
messages needs to be mangled to property-compatible form
(in the patch below, the only transformation replaced spaces ' ' with
underscores '_',
but it needs to be generalized).
I agree with you.
Given the number of issues PropertyResourceBundle introduces, and the number
of
services it provides (parsing property-format and constructing in-memory
hashmap),
I think that it would be easier to reimplement the functionality without
using PropertyResourceBundle,
and change the storage on-disk file format to allow unmangled messages be
the keys.
In conclusion there are my suggestions for VM's internationalization:
1. Extend log4cxx::helpers::PropertyResourceBundle class which should
allow lazy (on demand) load of properties.
2. Extend log4cxx::helpers::Properties class to allow string with
spaces as a key.
3. Choose model:
a. _(message key) – localizable ; message – not localizable
b. message key – localizable ; – not localizable.
4. Decide between two variants: printf format specifications or
{number} should be used inside message pattern for parameters.
Thanks Dmitry.
===
From: Salikh Zakirov [EMAIL PROTECTED]
Date: Thu, 13 Jul 2006 12:06:05 +0400
Subject: [PATCH] Dummy l10n implemenation based on Log4cxx
---
vm/include/l10n.h | 31 +++
vm/port/include/loggerstring.h |9 +
vm/vmcore/src/init/l10n.cpp| 66
vm/vmcore/src/init/vm_main.cpp |2 +
4 files changed, 108 insertions(+), 0 deletions(-)
diff --git a/vm/include/l10n.h b/vm/include/l10n.h
new file mode 100755
index 000..bb3edfe
--- /dev/null
+++ b/vm/include/l10n.h
@@ -0,0 +1,31 @@
+#ifndef _L10N_H
+#define _L10N_H
+
+#include string
+#include log4cxx/helpers/propertyresourcebundle.h
+#include log4cxx/helpers/exception.h
+#include wchar.h
+#include cxxlog.h
+
+extern log4cxx::helpers::ResourceBundlePtr
l10n_resource_bundle;
+
+inline const wchar_t* _(const wchar_t* message)
+{
+if (!l10n_resource_bundle) return message;
+try {
+wchar_t* mangled = wcsdup(message);
+wchar_t* c = mangled;
+while (*c) {
+if (*c == L' ') *c = L'_';
+c++;
+}
+std::wstring localized =
l10n_resource_bundle-getString(mangled);
+free(mangled);
+return wcsdup(localized.c_str()); // FIXME: leak
+} catch (log4cxx::helpers::MissingResourceException )
