Re: Lotj - languages other than java

[Peter]

Thanks Bishnu,

I plan to set up a public lookup service later this year, or early next 
over IPv6 (dependant on me completing the work on github), there's a 
beta release of the code available you can experiment with.

It will be possible to discover this lookup service dynamically using 
the global IPv6 multicast address FF0X::155 for Jini-announcement.

For DOS reasons, Jini-request will only be available for local network use.

http://www.iana.org/assignments/ipv6-multicast-addresses/ipv6-multicast-addresses.xhtml

Before I could make a lookup service publicly available, I've had to do 
a lot of work on addressing security issues:

  1. Serialization atomic input validation.
  2. Two new default methods in ServiceRegistrar so that clients can
     authenticate services, prior to deserializing their proxy's,
     Entry's and downloading or provisioning codebases.
  3. New discovery providers with updated cryptographic hash functions
     for secure discovery and added input validation for
     deserialization of registrar proxy.
  4. Dynamic granting of DownloadPermission and
     DeserializationPermission (similar to deserialization
     whitelisting) to authenticated services (to allow their proxy to
     be deserialized / downloaded).
  5. Allow listing of Permissions within jar files, for smart proxy's
     and have these granted dynamically if service authentication succeeds.
  6. Update encryption cyphers and invocation constraints.

Existing River / Jini deployments will not be able to contact this 
lookup service, in the event they could, they cannot satisfy the 
ConfidentialityStrength invocation constraints either unfortunately.  On 
local networks security could be relaxed sufficiently to allow 
compatibility with an existing River deployment, but public lookup 
services should not make that compromise.

The Serial form of objects are compatible with River.

This is all beta, new API may change if adopted by River.

Regards,

Peter.

On 6/07/2016 3:36 PM, Bishnu Gautam wrote:
Thanks Peter for your detail explanation regarding River on Internet.I will go 
through it. By the way do you have any publicly available Lookup service to 
which I can register my proxy service and this proxy service will refer my web 
server by which I could download my rest of the actual service codes (fully 
implemented) from the publicly available web server(in which my service codes 
are located). In fact I am trying to avail Lookup Server publicly so that the 
user could register their proxy services in my lookup server.Let me know if 
anybody already made it available for public use. Once we are able to do it we 
do not need to rely upon REST too and also do not need to tackle with NATing 
and packet filtering issues. I am looking whether there are any workable 
publicly available lookup services to which I can register my proxy.
RegardsBishnu


Bishnu Prasad Gautam


Date: Tue, 5 Jul 2016 17:46:14 +1000
From: j...@zeus.net.au
To: dev@river.apache.org
Subject: Re: Lotj - languages other than java

Thanks Bishnu,

Mark Brouwer originally pointed out many years ago, that while jini had
https jeri endpoints, there was no support to perform unicast discover
over https in LookupLocator discovery.

I have implemented that.

Now, you can have any number of publicly available lookup services, and
contact them using https unicast with LookupLocator,
ConstrainableLookupLocator and LookupLocatorDiscovery.  DNS-SD is an
obvious substitute for multicast discovery for clients behind firewalls,
but it hasn't been implemented yet.

With public lookup services, even when not discoverable with multicast
discovery (because your clients are behind firewalls), can register with
each other, using multicast discovery, allowing clients behind firewalls
to find others, so they only need to know a few permanent registrars to
find others, when they don't have access to multicast discovery.

Non public service clients (behind firewalls or NAT) can locate public
services.  Non public clients can listen behind NAT and firewalls, when
they register a listener with a lookup service, they will be notified of
new service registrations, because https jeri endpoints will keep the
connection (ports) open between a lookup service and clients behind
firewalls.

So lets say for example, you have an embedded client behind a firewall,
which is also a master for an IoT wireless local network and there are
several devices that will send a video stream or other data, then for
instance a public consumer service, could register with a lookup service
where the embedded client was listening, then be contacted and
authenticated by the embedded client and access a service directly from
the client to receive live data streams from that wireless network, this
may then republish the data accumulated from multiple such embedded
clients via a web site.  Clients behind NAT must initiate contact, only
then can they be contacted.

JERI multiplexes, so you can have 127 active service connections over
one connection between two nodes.

LetsEncrypt.org is a free certificate authority than can be utilised for
https jeri endpoints.  LetsEncrypt.org doesn't provide code
certificates, so they can't be used for signing jar files.

To avoid the expense of CA signed codebase certificates, https discovery
automatically grants DownloadPermission and DeserializationPermission
(which is required to allow proxy deserialization over a https jeri
endpoint with an InputValidation constraint), to an anoymous code signer
certificate and uri exchanged during the discovery process, but only if
authentication is successful.  The codebase jar file can also contain
the permissions it requires, so that these can be granted dynamically by
the client during proxy preparation.

In addition https jeri endpoint encryption cyphers have all been updated
to modern secure cyphers and support for insecure cyphers has been removed.

Default methods have been added to ServiceRegistrar, to allow the client
to authenticate services prior to retrieving proxy's and Entry's.
ProxyTrust verfiiers are no longer necessary as the proxy is obtained
directly from the service after authentication, rather than via a third
party.  The proxy is already trusted.

https://pfirmstone.github.io/river-internet/

I hope to get this accepted by River at some point, I figure that
creating a demonstration will assist the understanding process for other
developers, as I wasn't able to communicate effectively enough to avoid
strong resistance and criticism when I originally proposed it.

Regards,

Peter.


On 5/07/2016 1:33 PM, Bishnu Gautam wrote:
Hi Peter
It is great that you pointed out lookup locator issue in firewall and its 
potential solution. It would be great to see the developments in River in which 
they really focus to have lookup discovery beyond the firewall without 
requiring port forward and other demanding packet filtering techniques. Once 
this obstacle in River is crossed, I am pretty sure that there will be new 
paradigm shift in IoT or ICT technology. This technology has a tremendous 
potential. However, I never understand why River community never try to bring 
this issue on the first place. River in Internet would be the most wonderful 
solutions for millions of users around the world. Please think, discuss and try 
to work on it. It would be a great news for us.
RegardsBishnu


Bishnu Prasad Gautam


Date: Mon, 4 Jul 2016 18:37:25 +1000
From: j...@zeus.net.au
Subject: Re: Lotj - languages other than java
To: dev@river.apache.org
CC: si...@qcg.nl




Sim,

I'd like to see the project return to the days where we had a number of active 
committers working together on the same goals.

I've got a project on github, where I've continued work that was controversial, 
I'd like to bring this work back to the project if possible.  It has some minor 
breaking changes, but if backward compatibility was essential, it could be 
accommodated.

Changes:
* New secure discovery providers, including https among others, including added 
https protocol support for LookupLocator.  However since firewalls may not 
allow ipv6 multicast packets through, DNS-SD would be useful.
* IPv6 Discovery, global and local groups.
* Discovery V2 support added to LookupLocator's getRegistrar method.
* Updated encryption ciphers, removal of insecure ones.
* Deprecation of ProxyTrust et al.
* New default methods added to ServiceRegistrar to allow establising trust with 
a service, prior to obtaining a service proxy, or Entry's (works with both 
maven codebase provisioning and traditional codebase downloads).
* Input validation for untrusted serial data.
* META-INF/permissions.perm files list permissions required by codebase, 
accessible from ClassLoader using mixin interface.

I recall you had a need for a SocketFactory in LookupLocator, at that time 
LookupLocator only used discovery v1, which was deprecated, I'd like to include 
a way to enable SocketFactory support in discovery V2.  Note that LookupLocator 
isn't serialized during discovery.

While the River codebase was a little difficult to understand at first, I find 
it's quite easy to work with now.

While I'm a critic of Rivers flaws, addressing th em is straight forward, the 
greatest challenge is convincing everyone that flaws exist, or that they need 
addressing, there's still plenty of good stuff left worth saving.

Regards,

Peter.


Sent from my Samsung device.

    Include original message
---- Original message ----
From: Peter<j...@zeus.net.au>
Sent: 01/07/2016 04:35:16 pm
To: dev@river.apache.org<dev@river.apache.org>
Subject: Re: Lotj - languages other than java





Thanks Sim,

These are all good questions we need to consider.

I like the model of micro services where each service is responsible for 
implementing its own back end persistence and state.  Do you consider a 
microservice to be web based?

I have an implementation of discovery using multicast ipv6.  However for 
firewalls with limited open ports such as https over ipv6, we have JERI https 
endpoints, but no discovery, DNS-SD is a good discovery alternative waiting to 
be implemented.

For my own environment I will be adopting ipv6 , the global address space and 
autoconfiguration solve many problems that users experience with ipv4 today.

I admit the locked down api caused me frustration, but I think it's clear now 
that we need a process for managing api evolution.

Complexity - The proxy preparation api tries to determine trust after 
downloading untrusted code and deserialization of unverified data.  As gadget 
attacks demonstrate, too little too late at great complexity.  This was an 
attempt to bolt security onto the existing lookup service.

JERI is good, method constraints are good, proxy trust is obsolete.  River's 
current ssl and https JERI endpoints need to be brought up to date as they're 
no longer secure.  I've already done this work externally, it can be donated 
when appropriate for the project.

If we address security issues, we can provide a secure alternative to RMI  
Oracle has chosen 'whack a mole' security for Serialization, rather than 
address some fundamental design flaws with ObjectInputStream, for this reason, 
authentication of the source must occur prior to accepting serial data.  
Despite common belief, white listing isn't a completely secure solution and 
adds conplexity as it's too fine grained.

For multi language support, this would limit the type system, but then, there's 
a lot that can be done with strings, primitive types and byte arrays.  This 
doesn't have to limit java service types, I think language support should be 
something determined during lookup, so we don't limit the type systems of more 
powerful languages to primitives.

Looking at most Entry's used for lookup, most fields are strings and integers.  
If you look at the way lookup searches are implemented, an entry is represented 
by a string name and each field is a tuple name value pair.

I think a ground up redesign of the lookup service, would address language 
compatibility as well as complexity and security.

In other words, recognise the need for a lookup&   registration protocol, as well 
as discovery, after that, the service&   client should be able to negotiate  
whatever rpc protocol they have in common and to do that, we'll also need a connection 
negotiation protocol.  We could write specifications for these protocols.  This way we 
could allow any language/ platform to register and provide services.  The code for 
lookup would not be downloaded as Reggie is now, it would be protocol, rather than 
proxy based.  This would also fit well with IoT.

Meanwhile we can still support existing java based services.

Thoughts?

Peter.

Sent from my Samsung device.

    Include original message
---- Original message ----
From: Simon IJskes - QCG<si...@qcg.nl>
Sent: 30/06/2016 06:22:30 pm
To: dev@river.apache.org
Subject: Re: Lotj - languages other than java

If you solve the 'barrier' of the service discovery, do you also want to
provide universal access to the java services in the form of microservices?

It is doable to take any 'more used' service discovery solution and use
this as the river discovery. To introduce a level of abstraction with
the same primitives as the current river discovery mechanism offers.

River would then have adapted a more common discovery mechanism.

Next thing that we should decide, is how far do we go into universality.
I see univeral type systems, different serialisation plugins on the horizon.

The biggest showstopper for me was the API compatibility. In order to
make any progress we need a more agile process for modifing the API

If we leave compatibility behind us, we could ask our selfs, what
benefit are we providing for the users? What can we introduce that does
not duplicate what is already in the market For a java developer, i
think there is no need to convince, they can see benefits in just having
a java API to program against. We need to think about the environment
where java receives a lot of 'non-love', how we can create a 'whow, java
isn't all that bad, look at that easy solution' experience.

I think that river lost the spot it could have, as a java only solution
to JSON, XMLRPC, SOAP, etc libraries for java. From a helicopter view,
what does it do? Whe provide secure RPC, with discovery and scaling. And
we make it hard to use.

G. Simon


On 30-06-16 05:37, Peter wrote:
Currently with River, you need java to participate.  Other languages can 
provide services, but you need a jvm to participate.

Most of discovery is language agnostic, so any language can participate in 
discovery.

The major limitation for other languages is the lookup service.  Security 
issues and complexity also relate to the lookup service.

My thoughts are that a lookup service that performs search and registration, 
but provides a language independent  and secure means of contacting service 
providers would be beneficial.

Anyone interested in discussing further?

Regards,

Peter.


Sent from my Samsung device.