Re: Future plans
To be honest, I've got no interest in Internet-visible services or discovery. Not that I want to discourage other people working on them. If that's your itch, then scratch away! :-) Greg's new wiki page said one of the things that particularly hits home with me; If it were easier to get up and running... Absolutely, that's one of the things that I want to get sorted. I've spoken about it before and usually been told that it's the job of downstream projects to make 'stuff' easier. I kind of resent that attitude. There is no denying that there is a lot of truely brilliant stuff in downstream projects, that's no reason to leave using vanilla River as such a headache though. Other things that come to mind are the Gradle build and the distributed transaction manager. We should probably start thinking about a new roadmap as well. Yes, this is another vision thing, but now we're a TLP I think we're expected to have one. ;-) Cheers, Tom On Mon, Aug 1, 2011 at 1:10 AM, Peter Firmstone j...@zeus.net.au wrote: Patricia Shanahan wrote: On 7/31/2011 1:43 AM, Peter Firmstone wrote: ... * TaskManager - improve concurrency and remove the dependency on Task.runAfter() in River code. ... I'm playing with implementing a subclass of ThreadPoolExecutor that modifies the number of threads based on the mean number of tasks over some recent time period. That way, the number of threads will not be a long term bottleneck, but threads will not be created and destroyed for short-lived bursts. I have an on-going concern, for all performance and concurrency work in River, that we have neither benchmarks nor access to real installations to instrument and observe. I'm having the same issue, I've noticed some Sun T2000's becoming available on ebay for around $1000, 32GB ram, 8 cores which appear as 32 cpu's in the system. The only problem might be Oracle's licensing, but this is the cheapest most relevant hardware I can find at present. Object based benchmarks should be easy enough to create using MultitreadedTC (included in test/lib), but this doesn't apply to integration testing and I suspect the current qa suite might have some concurrency issues too. Peter.
Future plans
Just thought I'd go over the ideas, thoughts and TODO's that come to mind and get some feedback about what others are thinking and what tasks they see as important. There's plenty of work for those so inclined and generous with time. Brief Summary: * TaskManager - improve concurrency and remove the dependency on Task.runAfter() in River code. * The Surrogate Project. * Providing Services over the Internet: o NAT Traversal + UDT (UDP Based Data Transfer) http://udt.sourceforge.net/ - a NAT friendly alternative to TCP. + STUN, TURN, NAT-PMP, UPnP o DNS-SRV LookupDiscovery (discovering lookup services in internet domains using DNS). o DGC (Distributed Garbage Collection) investigating use of Secure Endpoints? o StreamServiceRegistrar - delayed unmarshalling, client side filtering and Javaspace MatchSet like result handling using ResultStream, to address some of the long term criticisms of ServiceRegistrar. * SecurityManager and Policy o River-323 ConcurrentDynamicPolicy - existing policy implementations cause multi threading lock contention (almost complete, just needs to be tested against the current trunk and merged). + River-249 Added support for umbrella grant's. o Permission Revocation (Framework implemented, requires standardization). + Delegate's - use Li Gong's method guard pattern to encapsulate Socket's, Streams etc. + DelegatePermission - to encapsulate an existing permission that allows references to security sensitive objects to escape. + Requires support from the SecurityManager, to check all ProtectionDomain's in the AccessControlContext for a DelegatePermission or it's candidate (the Permission encapsulated by a DelegatePermission). + Requires support from a RevocablePolicy, to remove a DelegatePermission (or other existing Permission that doesn't let references escape) from the policy. o InternetSecurityManager - support for caching repeated permission for each AccessControlContext. o SecurityPolicyService - Allow local Policy's to be updated by subscribing to a Service using secure Endpoint's and administrator Subject's, to simplify distributed policy maintenance and replication. This is in addition to policy files and dynamic grant's to proxy's. + Requires support from the Policy implementation. + Utilized and improved Apache Harmony File Policy Parser implementation + This isn't for dynamic grant's to Proxy's, but may be used to modify who (Subject) can make a dynamic grant. + Utilizes existing policy file syntax. + Allows granting of DownloadPermission to Certificate[] signers to prevent proxy unmarshalling DOS attacks. * River-32 Jini Lookup, Discovery and Join Test Kit - Get this codebase working again. * River-279 - Create a subproject called Jini (no longer trademarked) to manage the Jini Specifications? * Investigate conversion script for a Maven or Gradle build. * Separate JVM for isolation of downloaded code, to sandbox unauthenticated services. Cheers, Peter.
Re: Future plans
On 7/31/2011 1:43 AM, Peter Firmstone wrote: ... * TaskManager - improve concurrency and remove the dependency on Task.runAfter() in River code. ... I'm playing with implementing a subclass of ThreadPoolExecutor that modifies the number of threads based on the mean number of tasks over some recent time period. That way, the number of threads will not be a long term bottleneck, but threads will not be created and destroyed for short-lived bursts. I have an on-going concern, for all performance and concurrency work in River, that we have neither benchmarks nor access to real installations to instrument and observe. Patricia
Re: Future plans
Patricia Shanahan wrote: On 7/31/2011 1:43 AM, Peter Firmstone wrote: ... * TaskManager - improve concurrency and remove the dependency on Task.runAfter() in River code. ... I'm playing with implementing a subclass of ThreadPoolExecutor that modifies the number of threads based on the mean number of tasks over some recent time period. That way, the number of threads will not be a long term bottleneck, but threads will not be created and destroyed for short-lived bursts. I have an on-going concern, for all performance and concurrency work in River, that we have neither benchmarks nor access to real installations to instrument and observe. I'm having the same issue, I've noticed some Sun T2000's becoming available on ebay for around $1000, 32GB ram, 8 cores which appear as 32 cpu's in the system. The only problem might be Oracle's licensing, but this is the cheapest most relevant hardware I can find at present. Object based benchmarks should be easy enough to create using MultitreadedTC (included in test/lib), but this doesn't apply to integration testing and I suspect the current qa suite might have some concurrency issues too. Peter.
Re: Oh, no, the vision thing. Was: Re: Future plans
Greg Trasuk wrote: Inspired by Peter's post below, I've begun to elucidate some thoughts on where I see Jini/River at http://wiki.apache.org/river/JavaBasedSOA You've got my vote for the Surrogate SOA Adapter pattern. Cheers, Peter. For some reaon it won't let me attach a graphic. (You are not allowed to do AttachFile on this page). Any ideas? Maybe this should have it's own page on the river website outside of the wiki. I'd like to see some graphics explaining service dependencies too, along the lines that Dennis has done with Rio. Cheers, Peter. Cheers, Greg Trasuk. On Sun, 2011-07-31 at 04:43, Peter Firmstone wrote: Just thought I'd go over the ideas, thoughts and TODO's that come to mind and get some feedback about what others are thinking and what tasks they see as important. There's plenty of work for those so inclined and generous with time. Brief Summary: * TaskManager - improve concurrency and remove the dependency on Task.runAfter() in River code. * The Surrogate Project. * Providing Services over the Internet: o NAT Traversal + UDT (UDP Based Data Transfer) http://udt.sourceforge.net/ - a NAT friendly alternative to TCP. + STUN, TURN, NAT-PMP, UPnP o DNS-SRV LookupDiscovery (discovering lookup services in internet domains using DNS). o DGC (Distributed Garbage Collection) investigating use of Secure Endpoints? o StreamServiceRegistrar - delayed unmarshalling, client side filtering and Javaspace MatchSet like result handling using ResultStream, to address some of the long term criticisms of ServiceRegistrar. * SecurityManager and Policy o River-323 ConcurrentDynamicPolicy - existing policy implementations cause multi threading lock contention (almost complete, just needs to be tested against the current trunk and merged). + River-249 Added support for umbrella grant's. o Permission Revocation (Framework implemented, requires standardization). + Delegate's - use Li Gong's method guard pattern to encapsulate Socket's, Streams etc. + DelegatePermission - to encapsulate an existing permission that allows references to security sensitive objects to escape. + Requires support from the SecurityManager, to check all ProtectionDomain's in the AccessControlContext for a DelegatePermission or it's candidate (the Permission encapsulated by a DelegatePermission). + Requires support from a RevocablePolicy, to remove a DelegatePermission (or other existing Permission that doesn't let references escape) from the policy. o InternetSecurityManager - support for caching repeated permission for each AccessControlContext. o SecurityPolicyService - Allow local Policy's to be updated by subscribing to a Service using secure Endpoint's and administrator Subject's, to simplify distributed policy maintenance and replication. This is in addition to policy files and dynamic grant's to proxy's. + Requires support from the Policy implementation. + Utilized and improved Apache Harmony File Policy Parser implementation + This isn't for dynamic grant's to Proxy's, but may be used to modify who (Subject) can make a dynamic grant. + Utilizes existing policy file syntax. + Allows granting of DownloadPermission to Certificate[] signers to prevent proxy unmarshalling DOS attacks. * River-32 Jini Lookup, Discovery and Join Test Kit - Get this codebase working again. * River-279 - Create a subproject called Jini (no longer trademarked) to manage the Jini Specifications? * Investigate conversion script for a Maven or Gradle build. * Separate JVM for isolation of downloaded code, to sandbox unauthenticated services. Cheers, Peter.
