[jira] [Commented] (FELIX-3624) Performance tuning solution for BundleRepository/ResolverImpl
[
https://issues.apache.org/jira/browse/FELIX-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491834#comment-13491834
]
Jarek Gawor commented on FELIX-3624:
I think there might be a potential problem with the patch with the
searchResources(Requirement, Set) function. The function is supposed to search
for matching resources within a specified set of resources. With caching
enabled that means the m_capabilitiesCache will be checked first - a cache of
local and remote resources only. Once a matching resource is found in the cache
it will be checked against the specified set. That's good but there is at least
one case where the cache might not contain a resource in the specified set. For
example, the m_addedSet - which user can add resources to before resolve() is
called. So in this case, with caching, searchResources() would return null
instead of a matching resource from the m_addedSet.
Performance tuning solution for BundleRepository/ResolverImpl
-
Key: FELIX-3624
URL: https://issues.apache.org/jira/browse/FELIX-3624
Project: Felix
Issue Type: Improvement
Components: Bundle Repository (OBR)
Environment: Geronimo3 + BundleRepository 1.6.6
Reporter: SheldonShao
Priority: Critical
Attachments: Felix_ResolverImpl_AfterTuning.png,
Felix_ResolverImpl.png, repo_for_resolver_perf.xml, ResolverImpl.diff,
ResolverImpl.java, ResolverImplTest.diff
There are too many numebers of method calling about checking whether
capabities are matched with requiment in ResolverImpl.searchResources.
This is a performance issue of ResolverImpl.resolve.
If it creates a capabity to resource+capabity mapping first. Then leverage
this cache to do requirment matching. It will get better performance.
Here is part of code. More details please check attachments.
private final Map m_capabilitiesCache = new HashMap(8192);
private String getKey(String filter, String prefix) {
if (filter != null) {
int index = filter.indexOf(prefix);
if (index 0) {
int end = filter.indexOf(SUFFIX, index +
prefix.length());
if (end index) {
return filter.substring(index, end);
}
}
}
return null;
}
private String getKey(Requirement requirement) {
String key = null;
String name = requirement.getName();
String filter = requirement.getFilter();
if (Capability.BUNDLE.equals(name)) {
key = getKey(filter, PREFIX_SYMBOLICNAME);
} else if (Capability.PACKAGE.equals(name)) {
key = getKey(filter, PREFIX_PACKAGE);
} else if (Capability.SERVICE.equals(name)) {
key = getKey(filter, PREFIX_SERVICE);
} else if (Capability.FRAGMENT.equals(name)) {
key = getKey(filter, PREFIX_HOST);
} else {
key = PREFIX_CAPABILITY + name;
}
return key;
}
private static final String PREFIX_SYMBOLICNAME = symbolicname=;
private static final String PREFIX_PACKAGE = package=;
private static final String PREFIX_SERVICE = service=;
private static final String PREFIX_HOST = host=;
private static final String PREFIX_CAPABILITY = capability=;
private static final char SUFFIX = ')';
private void initCache(Capability[] capabilities, Resource resource) {
Capability cap;
Map properties;
String name;
for (int j = 0; j capabilities.length; j++) {
cap = capabilities[j];
String key = null;
properties = cap.getPropertiesAsMap();
name = cap.getName();
if (Capability.BUNDLE.equals(name)) {
key = PREFIX_SYMBOLICNAME +
properties.get(symbolicname);
} else if (Capability.PACKAGE.equals(name)) {
key = PREFIX_PACKAGE +
properties.get(package);
} else if (Capability.SERVICE.equals(name)) {
key = PREFIX_SERVICE +
properties.get(service);
} else if (Capability.FRAGMENT.equals(name)) {
key = PREFIX_HOST + properties.get(host);
} else {
key = PREFIX_CAPABILITY + name;
}
[jira] [Commented] (FELIX-3624) Performance tuning solution for BundleRepository/ResolverImpl
[
https://issues.apache.org/jira/browse/FELIX-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13432972#comment-13432972
]
SheldonShao commented on FELIX-3624:
Uploaded patch files for ResolverImpl.java ResolverImplTest.java
Also I added a test case for this performance tuning and a data file
repo_for_resolver_perf.xml.
Here is the result of the benchmark,
Has cache:2012 ms
No cache:5643 ms
Performance tuning solution for BundleRepository/ResolverImpl
-
Key: FELIX-3624
URL: https://issues.apache.org/jira/browse/FELIX-3624
Project: Felix
Issue Type: Improvement
Components: Bundle Repository (OBR)
Environment: Geronimo3 + BundleRepository 1.6.6
Reporter: SheldonShao
Priority: Critical
Attachments: Felix_ResolverImpl_AfterTuning.png,
Felix_ResolverImpl.png, repo_for_resolver_perf.xml, ResolverImpl.diff,
ResolverImpl.java, ResolverImplTest.diff
There are too many numebers of method calling about checking whether
capabities are matched with requiment in ResolverImpl.searchResources.
This is a performance issue of ResolverImpl.resolve.
If it creates a capabity to resource+capabity mapping first. Then leverage
this cache to do requirment matching. It will get better performance.
Here is part of code. More details please check attachments.
private final Map m_capabilitiesCache = new HashMap(8192);
private String getKey(String filter, String prefix) {
if (filter != null) {
int index = filter.indexOf(prefix);
if (index 0) {
int end = filter.indexOf(SUFFIX, index +
prefix.length());
if (end index) {
return filter.substring(index, end);
}
}
}
return null;
}
private String getKey(Requirement requirement) {
String key = null;
String name = requirement.getName();
String filter = requirement.getFilter();
if (Capability.BUNDLE.equals(name)) {
key = getKey(filter, PREFIX_SYMBOLICNAME);
} else if (Capability.PACKAGE.equals(name)) {
key = getKey(filter, PREFIX_PACKAGE);
} else if (Capability.SERVICE.equals(name)) {
key = getKey(filter, PREFIX_SERVICE);
} else if (Capability.FRAGMENT.equals(name)) {
key = getKey(filter, PREFIX_HOST);
} else {
key = PREFIX_CAPABILITY + name;
}
return key;
}
private static final String PREFIX_SYMBOLICNAME = symbolicname=;
private static final String PREFIX_PACKAGE = package=;
private static final String PREFIX_SERVICE = service=;
private static final String PREFIX_HOST = host=;
private static final String PREFIX_CAPABILITY = capability=;
private static final char SUFFIX = ')';
private void initCache(Capability[] capabilities, Resource resource) {
Capability cap;
Map properties;
String name;
for (int j = 0; j capabilities.length; j++) {
cap = capabilities[j];
String key = null;
properties = cap.getPropertiesAsMap();
name = cap.getName();
if (Capability.BUNDLE.equals(name)) {
key = PREFIX_SYMBOLICNAME +
properties.get(symbolicname);
} else if (Capability.PACKAGE.equals(name)) {
key = PREFIX_PACKAGE +
properties.get(package);
} else if (Capability.SERVICE.equals(name)) {
key = PREFIX_SERVICE +
properties.get(service);
} else if (Capability.FRAGMENT.equals(name)) {
key = PREFIX_HOST + properties.get(host);
} else {
key = PREFIX_CAPABILITY + name;
}
List caps = (List) m_capabilitiesCache.get(key);
if (caps == null) {
caps = new ArrayList(2);
m_capabilitiesCache.put(key, caps);
}
caps.add(new ResourceCapabilityImpl(resource, cap));
}
}
private void initCache(Resource[] locals) {
Resource resource;
for (int i = 0; i locals.length; i++) {
[jira] [Commented] (FELIX-3624) Performance tuning solution for BundleRepository/ResolverImpl
[
https://issues.apache.org/jira/browse/FELIX-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431657#comment-13431657
]
Felix Meschberger commented on FELIX-3624:
--
First, thanks for reporting and analyzing. I will not comment on the results or
your changes because I don't know that part of the code too deeply.
But you contribution as it stands can unfortunately not be accepted: For one
you forgot to check Grant license to ASF for inclusion in ASF works when
uploading your ResolverImpl.java file and second, I (personally) would prefer
to get a diff against current trunk instead of a full file, where it is not
easily visible what the changes actually are.
May I ask you to provide a proper patch and check the grant button ? Thanks.
Performance tuning solution for BundleRepository/ResolverImpl
-
Key: FELIX-3624
URL: https://issues.apache.org/jira/browse/FELIX-3624
Project: Felix
Issue Type: Improvement
Components: Bundle Repository (OBR)
Environment: Geronimo3 + BundleRepository 1.6.6
Reporter: SheldonShao
Priority: Critical
Attachments: Felix_ResolverImpl.png,
Felix_ResolverImpl_AfterTuning.png, ResolverImpl.java
There are too many numebers of method calling about checking whether
capabities are matched with requiment in ResolverImpl.searchResources.
This is a performance issue of ResolverImpl.resolve.
If it creates a capabity to resource+capabity mapping first. Then leverage
this cache to do requirment matching. It will get better performance.
Here is part of code. More details please check attachments.
private final Map m_capabilitiesCache = new HashMap(8192);
private String getKey(String filter, String prefix) {
if (filter != null) {
int index = filter.indexOf(prefix);
if (index 0) {
int end = filter.indexOf(SUFFIX, index +
prefix.length());
if (end index) {
return filter.substring(index, end);
}
}
}
return null;
}
private String getKey(Requirement requirement) {
String key = null;
String name = requirement.getName();
String filter = requirement.getFilter();
if (Capability.BUNDLE.equals(name)) {
key = getKey(filter, PREFIX_SYMBOLICNAME);
} else if (Capability.PACKAGE.equals(name)) {
key = getKey(filter, PREFIX_PACKAGE);
} else if (Capability.SERVICE.equals(name)) {
key = getKey(filter, PREFIX_SERVICE);
} else if (Capability.FRAGMENT.equals(name)) {
key = getKey(filter, PREFIX_HOST);
} else {
key = PREFIX_CAPABILITY + name;
}
return key;
}
private static final String PREFIX_SYMBOLICNAME = symbolicname=;
private static final String PREFIX_PACKAGE = package=;
private static final String PREFIX_SERVICE = service=;
private static final String PREFIX_HOST = host=;
private static final String PREFIX_CAPABILITY = capability=;
private static final char SUFFIX = ')';
private void initCache(Capability[] capabilities, Resource resource) {
Capability cap;
Map properties;
String name;
for (int j = 0; j capabilities.length; j++) {
cap = capabilities[j];
String key = null;
properties = cap.getPropertiesAsMap();
name = cap.getName();
if (Capability.BUNDLE.equals(name)) {
key = PREFIX_SYMBOLICNAME +
properties.get(symbolicname);
} else if (Capability.PACKAGE.equals(name)) {
key = PREFIX_PACKAGE +
properties.get(package);
} else if (Capability.SERVICE.equals(name)) {
key = PREFIX_SERVICE +
properties.get(service);
} else if (Capability.FRAGMENT.equals(name)) {
key = PREFIX_HOST + properties.get(host);
} else {
key = PREFIX_CAPABILITY + name;
}
List caps = (List) m_capabilitiesCache.get(key);
if (caps == null) {
caps = new ArrayList(2);
m_capabilitiesCache.put(key, caps);
[jira] [Commented] (FELIX-3624) Performance tuning solution for BundleRepository/ResolverImpl
[
https://issues.apache.org/jira/browse/FELIX-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431755#comment-13431755
]
Marcel Offermans commented on FELIX-3624:
-
Also, it would be nice to include some kind of test that performs the
measurements that you have been using to write this solution. That way we can
easily evaluate what the original problem was and how much better it has become.
Performance tuning solution for BundleRepository/ResolverImpl
-
Key: FELIX-3624
URL: https://issues.apache.org/jira/browse/FELIX-3624
Project: Felix
Issue Type: Improvement
Components: Bundle Repository (OBR)
Environment: Geronimo3 + BundleRepository 1.6.6
Reporter: SheldonShao
Priority: Critical
Attachments: Felix_ResolverImpl.png,
Felix_ResolverImpl_AfterTuning.png, ResolverImpl.java
There are too many numebers of method calling about checking whether
capabities are matched with requiment in ResolverImpl.searchResources.
This is a performance issue of ResolverImpl.resolve.
If it creates a capabity to resource+capabity mapping first. Then leverage
this cache to do requirment matching. It will get better performance.
Here is part of code. More details please check attachments.
private final Map m_capabilitiesCache = new HashMap(8192);
private String getKey(String filter, String prefix) {
if (filter != null) {
int index = filter.indexOf(prefix);
if (index 0) {
int end = filter.indexOf(SUFFIX, index +
prefix.length());
if (end index) {
return filter.substring(index, end);
}
}
}
return null;
}
private String getKey(Requirement requirement) {
String key = null;
String name = requirement.getName();
String filter = requirement.getFilter();
if (Capability.BUNDLE.equals(name)) {
key = getKey(filter, PREFIX_SYMBOLICNAME);
} else if (Capability.PACKAGE.equals(name)) {
key = getKey(filter, PREFIX_PACKAGE);
} else if (Capability.SERVICE.equals(name)) {
key = getKey(filter, PREFIX_SERVICE);
} else if (Capability.FRAGMENT.equals(name)) {
key = getKey(filter, PREFIX_HOST);
} else {
key = PREFIX_CAPABILITY + name;
}
return key;
}
private static final String PREFIX_SYMBOLICNAME = symbolicname=;
private static final String PREFIX_PACKAGE = package=;
private static final String PREFIX_SERVICE = service=;
private static final String PREFIX_HOST = host=;
private static final String PREFIX_CAPABILITY = capability=;
private static final char SUFFIX = ')';
private void initCache(Capability[] capabilities, Resource resource) {
Capability cap;
Map properties;
String name;
for (int j = 0; j capabilities.length; j++) {
cap = capabilities[j];
String key = null;
properties = cap.getPropertiesAsMap();
name = cap.getName();
if (Capability.BUNDLE.equals(name)) {
key = PREFIX_SYMBOLICNAME +
properties.get(symbolicname);
} else if (Capability.PACKAGE.equals(name)) {
key = PREFIX_PACKAGE +
properties.get(package);
} else if (Capability.SERVICE.equals(name)) {
key = PREFIX_SERVICE +
properties.get(service);
} else if (Capability.FRAGMENT.equals(name)) {
key = PREFIX_HOST + properties.get(host);
} else {
key = PREFIX_CAPABILITY + name;
}
List caps = (List) m_capabilitiesCache.get(key);
if (caps == null) {
caps = new ArrayList(2);
m_capabilitiesCache.put(key, caps);
}
caps.add(new ResourceCapabilityImpl(resource, cap));
}
}
private void initCache(Resource[] locals) {
Resource resource;
for (int i = 0; i locals.length; i++) {
resource = locals[i];
Capability[] capabilities =
[jira] [Commented] (FELIX-3624) Performance tuning solution for BundleRepository/ResolverImpl
[
https://issues.apache.org/jira/browse/FELIX-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431757#comment-13431757
]
SheldonShao commented on FELIX-3624:
I am new for this. I will check Grant license to ASF for inclusion in ASF
works later.
And also I am going to refine the code and provide what I changed and test
result.
Thanks guys.
Performance tuning solution for BundleRepository/ResolverImpl
-
Key: FELIX-3624
URL: https://issues.apache.org/jira/browse/FELIX-3624
Project: Felix
Issue Type: Improvement
Components: Bundle Repository (OBR)
Environment: Geronimo3 + BundleRepository 1.6.6
Reporter: SheldonShao
Priority: Critical
Attachments: Felix_ResolverImpl.png,
Felix_ResolverImpl_AfterTuning.png, ResolverImpl.java
There are too many numebers of method calling about checking whether
capabities are matched with requiment in ResolverImpl.searchResources.
This is a performance issue of ResolverImpl.resolve.
If it creates a capabity to resource+capabity mapping first. Then leverage
this cache to do requirment matching. It will get better performance.
Here is part of code. More details please check attachments.
private final Map m_capabilitiesCache = new HashMap(8192);
private String getKey(String filter, String prefix) {
if (filter != null) {
int index = filter.indexOf(prefix);
if (index 0) {
int end = filter.indexOf(SUFFIX, index +
prefix.length());
if (end index) {
return filter.substring(index, end);
}
}
}
return null;
}
private String getKey(Requirement requirement) {
String key = null;
String name = requirement.getName();
String filter = requirement.getFilter();
if (Capability.BUNDLE.equals(name)) {
key = getKey(filter, PREFIX_SYMBOLICNAME);
} else if (Capability.PACKAGE.equals(name)) {
key = getKey(filter, PREFIX_PACKAGE);
} else if (Capability.SERVICE.equals(name)) {
key = getKey(filter, PREFIX_SERVICE);
} else if (Capability.FRAGMENT.equals(name)) {
key = getKey(filter, PREFIX_HOST);
} else {
key = PREFIX_CAPABILITY + name;
}
return key;
}
private static final String PREFIX_SYMBOLICNAME = symbolicname=;
private static final String PREFIX_PACKAGE = package=;
private static final String PREFIX_SERVICE = service=;
private static final String PREFIX_HOST = host=;
private static final String PREFIX_CAPABILITY = capability=;
private static final char SUFFIX = ')';
private void initCache(Capability[] capabilities, Resource resource) {
Capability cap;
Map properties;
String name;
for (int j = 0; j capabilities.length; j++) {
cap = capabilities[j];
String key = null;
properties = cap.getPropertiesAsMap();
name = cap.getName();
if (Capability.BUNDLE.equals(name)) {
key = PREFIX_SYMBOLICNAME +
properties.get(symbolicname);
} else if (Capability.PACKAGE.equals(name)) {
key = PREFIX_PACKAGE +
properties.get(package);
} else if (Capability.SERVICE.equals(name)) {
key = PREFIX_SERVICE +
properties.get(service);
} else if (Capability.FRAGMENT.equals(name)) {
key = PREFIX_HOST + properties.get(host);
} else {
key = PREFIX_CAPABILITY + name;
}
List caps = (List) m_capabilitiesCache.get(key);
if (caps == null) {
caps = new ArrayList(2);
m_capabilitiesCache.put(key, caps);
}
caps.add(new ResourceCapabilityImpl(resource, cap));
}
}
private void initCache(Resource[] locals) {
Resource resource;
for (int i = 0; i locals.length; i++) {
resource = locals[i];
Capability[] capabilities = resource.getCapabilities();
if
[jira] [Commented] (FELIX-3624) Performance tuning solution for BundleRepository/ResolverImpl
[
https://issues.apache.org/jira/browse/FELIX-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431806#comment-13431806
]
Richard S. Hall commented on FELIX-3624:
Thanks for looking into this.
I am hopeful that eventually we will address some of these issues by
re-implementing OBR using the new resolver subproject and then we could use the
capability indexing approach of the framework to speed up matching too.
Performance tuning solution for BundleRepository/ResolverImpl
-
Key: FELIX-3624
URL: https://issues.apache.org/jira/browse/FELIX-3624
Project: Felix
Issue Type: Improvement
Components: Bundle Repository (OBR)
Environment: Geronimo3 + BundleRepository 1.6.6
Reporter: SheldonShao
Priority: Critical
Attachments: Felix_ResolverImpl.png,
Felix_ResolverImpl_AfterTuning.png, ResolverImpl.java
There are too many numebers of method calling about checking whether
capabities are matched with requiment in ResolverImpl.searchResources.
This is a performance issue of ResolverImpl.resolve.
If it creates a capabity to resource+capabity mapping first. Then leverage
this cache to do requirment matching. It will get better performance.
Here is part of code. More details please check attachments.
private final Map m_capabilitiesCache = new HashMap(8192);
private String getKey(String filter, String prefix) {
if (filter != null) {
int index = filter.indexOf(prefix);
if (index 0) {
int end = filter.indexOf(SUFFIX, index +
prefix.length());
if (end index) {
return filter.substring(index, end);
}
}
}
return null;
}
private String getKey(Requirement requirement) {
String key = null;
String name = requirement.getName();
String filter = requirement.getFilter();
if (Capability.BUNDLE.equals(name)) {
key = getKey(filter, PREFIX_SYMBOLICNAME);
} else if (Capability.PACKAGE.equals(name)) {
key = getKey(filter, PREFIX_PACKAGE);
} else if (Capability.SERVICE.equals(name)) {
key = getKey(filter, PREFIX_SERVICE);
} else if (Capability.FRAGMENT.equals(name)) {
key = getKey(filter, PREFIX_HOST);
} else {
key = PREFIX_CAPABILITY + name;
}
return key;
}
private static final String PREFIX_SYMBOLICNAME = symbolicname=;
private static final String PREFIX_PACKAGE = package=;
private static final String PREFIX_SERVICE = service=;
private static final String PREFIX_HOST = host=;
private static final String PREFIX_CAPABILITY = capability=;
private static final char SUFFIX = ')';
private void initCache(Capability[] capabilities, Resource resource) {
Capability cap;
Map properties;
String name;
for (int j = 0; j capabilities.length; j++) {
cap = capabilities[j];
String key = null;
properties = cap.getPropertiesAsMap();
name = cap.getName();
if (Capability.BUNDLE.equals(name)) {
key = PREFIX_SYMBOLICNAME +
properties.get(symbolicname);
} else if (Capability.PACKAGE.equals(name)) {
key = PREFIX_PACKAGE +
properties.get(package);
} else if (Capability.SERVICE.equals(name)) {
key = PREFIX_SERVICE +
properties.get(service);
} else if (Capability.FRAGMENT.equals(name)) {
key = PREFIX_HOST + properties.get(host);
} else {
key = PREFIX_CAPABILITY + name;
}
List caps = (List) m_capabilitiesCache.get(key);
if (caps == null) {
caps = new ArrayList(2);
m_capabilitiesCache.put(key, caps);
}
caps.add(new ResourceCapabilityImpl(resource, cap));
}
}
private void initCache(Resource[] locals) {
Resource resource;
for (int i = 0; i locals.length; i++) {
resource = locals[i];
