I don't think we should mix the old client code with a new API. We
should keep the new client code separate from the server. Maybe even in
a different repo, as I think Fred suggested.
Le 4/26/2017 à 3:12 PM, Kirk Lund a écrit :
If we want to add a new API then I suggest we create new API packages and
leave the "cache" package as the old deprecated API for backwards
compatibility.
The new APIs and implementation could be separated into different geode
modules and packages (something like what follows):
org.apache.geode.api -- the main rich API (geode-api module and jar)
org.apache.geode.api.client -- the new thin client API (geode-api-client
module and jar)
org.apache.geode.core -- the main implementation of api (geode-core module
and jar which also currently contains the old API)
On Wed, Apr 26, 2017 at 9:41 AM, William Markito Oliveira <
william.mark...@gmail.com> wrote:
This is an awesome discussion and Jake's hitting all the right notes about
why JCache is a good idea! I've fought that fight in the past and lost it
but I'm happy to see it coming back...
What's really nice about Geode is that the functionalities and capabilities
are all there, they're just not that well exposed, known by others or
obscured by some decisions that doesn't apply anymore.
It's the same conversation about monitoring and statistics... All the
capability is there with JMX and Stats, but using an unknown custom format
or tool to display that data makes it not very appealing for OSS and
enterprise users that need workarounds and hacks to integrate with common
monitoring tools.
Refactoring API Client APIs, documentation and implementation of a new
Protocol, Reactive APIs, better integration with standard monitoring tools
- Sounds like good points for a 2.0 roadmap IMHO.
On Wed, Apr 26, 2017 at 10:28 AM, Jacob Barrett <jbarr...@pivotal.io>
wrote:
Wes,
Those are almost all administrative commands and have no place on the
client API. They belong on an administrative API or as I'm arguing a
series
of MBeans/JMX as it is already an established standard.
-Jake
On Wed, Apr 26, 2017 at 8:09 AM Wes Williams <wwilli...@pivotal.io>
wrote:
Now we're getting some precision. Let's talk about the "raw" Geode
proprietary bad ass API! Would that "raw" Geode proprietary bad ass
API
"raw"
Geode proprietary bad ass API that we're talking about be centered
around
the commands found here:
https://github.com/apache/geode/tree/rel/v1.1.1/geode-
core/src/main/java/org/apache/geode/management/internal/cli/commands
Or somewhere else?
*Wes Williams | Pivotal Advisory **Data Engineer*
781.606.0325
http://pivotal.io/big-data/pivotal-gemfire
On Tue, Apr 25, 2017 at 11:41 PM, Jacob Barrett <jbarr...@pivotal.io>
wrote:
Java and its community have standards for all of these issue so why
re-invent the wheel. The market doesn't want proprietary anymore,
they
want
standards and mobility.
Configuration of the server should happen through MBeans. You can
wrap
that
in gfsh for command line, REST for remote web based admin, use
JConsole
or
any other number of JMX based enterprise management tools. By using
MBeans
the server can easily expose new discovered services without the need
to
code specific gfsh commands, REST interfaces or APIs. There is no
reason
my
SDG can't be retooled to "discover" the configuration from these
MBeans
as
well rather than having to be touched every time we add or change
something. There are tools and books already written that
implementors
can
consult on MBeans. There isn't anything out there on gfsh commands.
If we want to play in the Java community, especially J2EE (the other
50%
of
Java that isn't Spring), then we had better have a JSR-107 answer no
matter
what the pain is to provide it. I can pull dozens of books of the
shelf
that teach me how to effectively use a JCache, how many can I pull
off
the
shelf that teach me Geode's API? How many engineers can I get
applications
form by saying "must have Geode API knowledge"? I can find people
with
JCache knowledge though. So from in implementor's perspective having
standards is a must. Now I don't think the JSR-107 interface should
be
the
root interface but rather a facade on the "raw" Geode proprietary bad
ass
API. That API should be 100% asynchronous (reactive, SEDA, whatever
the
current buzzword for asynchronous is today). Around that API we can
provide
facades for JSR 107, ConcurrentMap (our current yet not so well
behaving
API), List, Queue, etc. Maybe even JPA, JCA, etc. The thought of
putting
all those features into a single API makes my head exploded and they
don't
need to be like they are today.
On Tue, Apr 25, 2017 at 8:25 PM Wes Williams <wwilli...@pivotal.io>
wrote:
A couple of points to interact with John's points.
GFSH as API
We agree that GFSH is a DSL, and a really good and useful one. We
agree
that we don't want our API tied to GFSH syntax. I view the valuable
part
of
GemFire admin as the Java code underneath GFSH, or the "Commands."
For example, to create a JAVA API to "Create Region", why not
create a
Java API around CreateAlterDestroyRegionCommands? In this way,
GFSH
and
the
JAVA API share the same code. It seems too obvious yet I don't see
it
being
recommended. Why not? (Note: the hard-coded formatting would need
to
be
removed).
Once you have the Java/GFSH/REST API as common code, you then
refactor
it.
What's the objection to this approach? Once you open Java API's to
do
everything that GFSH does, then you have now unshackled the power
of
the
developer (the JAVA API) and the admin (GFSH API).
REST API
I've found that most don't want to use the Dev REST API because
it's
attached to a server rather than the cluster. HA?
*Wes Williams | Pivotal Advisory **Data Engineer*
781.606.0325 <(781)%20606-0325>
http://pivotal.io/big-data/pivotal-gemfire
On Tue, Apr 25, 2017 at 7:01 PM, Fred Krone <fkr...@pivotal.io>
wrote:
Good feedback.
This would use the new protocol. I should have mentioned that.
The original driver for this was the Region API needs either an
update
or
an alternative. Updating has a few drawbacks: Region wasn't
designed
with
open source in-mind and as Swap mentioned it is naturally tightly
coupled.
Members of the community are already working to update Region but
something
gets fixed and it breaks something for someone else. I think
it's
much
better to provide a new interface that implements the first part
of
JSR
107
(javax.cache) and get the ball rolling for the community and,
perhaps,
over
time deprecate Region (although that's not a primary objective).
A Java driver will probably get built regardless just to give the
new
protocol some legs. That driver also needs a decent caching
interface.
JSR
107 has already solved that part. So let's get started on it.
If
the
community wants to go the whole way and continue JSR 107
implementation
after that that's awesome. Functions can also be added, etc.
I intentionally did not mention anything about PCF as this
pertains
to
Geode itself as an open source offering and developer experience.
I'm
writing as a member of the community. Ie: I'm a developer who
would
like
to
add some caching to my application -- I can download either Geode
or
Hazelcast for free and right now it's a no brainer. Not that we
wouldn't
keep PCF in-mind but it's out of scope for this thread. I do
believe
getting started on a Java driver for the protocol and a
standardized
caching API are easily leveraged wins across the board.
On Tue, Apr 25, 2017 at 3:20 PM, Swapnil Bawaskar <
sbawas...@pivotal.io>
wrote:
I had looked at the JCache in the past and here are some of the
things
I
noted:
Naming convention: Geode's region is a Cache in JSR-107, and
Geode's
Cache
is a CacheManager. I think it would be too confusing to change
the
meaning
of cache. Also, how do you document this given that Cache means
different
things if you are talking JCache vs Geode.
The way to create a Cache in JSR-107 is:
Cache<Integer, Date> cache = manager.createCache(cacheName,
Configuration
c);
Where it is upto the implementation to extend Configuration.
Given
this,
users will not be able to switch from an existing
implementation
to
ours;
will have to write new code specially Configuration, making
callbacks
serializable etc.
JSR-107 will not be limited to the client. Server side
callbacks
like
CacheLoader, CacheListener etc. will need handle on jsr-107
“cache”.
JSR-107 supports features like an EntryProcessor, which is a
function
invoked atomically on an entry operation. We will have to make
invasive
changes to Geode to support this.
Given these, I don't think supporting JSR-107 is trivial.
On Tue, Apr 25, 2017 at 2:55 PM Dan Smith <dsm...@pivotal.io>
wrote:
What transport are you planning on using? REST, or the
current
binary
protocol? Or is this just a wrapper around the existing java
client
APIs?
If this about creating a new API, I agree with what John is
saying
that
we
need reduce the number of APIs we have to do the same thing
in
java.
It
seems especially confusing if we end up with different APIs
that
support
distinct features - like being able to create a region on the
server
with
this new API but only being able to invoke a function with
the
old
API.
The other thing I think we need to avoid is being able to do
things
from
the client (or gfsh, or xml, ...) that don't have a java API
on
the
server.
I'm assuming your getOrCreateRegion region is supposed behave
like
the
gfsh
command and update the cluster configuration? +++1 to Wes's
suggestion
have
a public API for executing these gfsh operations.
I really think we need to work on having *one* authoritative
public
API
that contains everything that we support on the server. The
protocols
we
support (REST, binary, ...) and the client drivers that use
those
protocols
should just be ways of accessing that API. The Java API on
the
server
right
now the closest thing we have to this, but there are things
you
can
do
with
gfsh and things you can do with the current client that have
no
java
API
equivalent. I think we really need to fix that!
Also, preferably we won't have to hand code a bunch of stuff
in
every
client driver and protocol whenever we add a new feature. For
example
if
were to add a geode function to invoke each new API we add,
that
new
API
would be accessible from REST, gfsh, the java client etc.
Later
we
could
add syntatic sugar to make the new API prettier, but if we
had
a
low
level
API that automatically exposed all new features that would
make
adding
new
features much less painful.
I do like the idea of adding a reactive API.
Supporting JSR-107 might be nice - but that could probably
be
written
just
as a wrapper around the current API without too much work. I
don't
think
we
should do anything for JSR-107 other than provide a JSR-107
compatible
wrapper - if someone wants additional geode specific features
they
should
drop into the existing API.
I also like the idea of a smaller client jar and
dependencies,
but
I
think
there is a huge amount of room for improvement in our
existing
client
just
by refactoring the code a bit more and shinking geode-core
into a
bare
minimum.
-Dan
On Mon, Apr 24, 2017 at 8:20 PM, Fred Krone <
fkr...@pivotal.io
wrote:
That's great, Wes. I will take a look. Thanks.
John -- good feedback to consider and I was hoping this
would
come
up.
"In my mind, there really are only 2 approaches... *Spring*
and
non-*Spring*,
or rather PCF and non-PCF, and since PCF is primarily based
on
Boot
(given
Microservices/applications are the new concurrency), then I
am
really
saying the same thing."
This would be cover non spring and attempt to give the
community
something
that had the same standardized experience as JSR 107 --
starting
with
the
Cache interface itself. Although we don't necessarily have
to
implement
Cache, it's method signatures are essentially a (non
spring)
caching
standard for Java developers at this point. We considered
full
blown
JSR
107 implementation but thought it was too robust for the
needs
mentioned
(that's not to say we couldn't get there incrementally). I
think
those
needs still exist open-source outside of PCF and don't
cannibalize.
On Mon, Apr 24, 2017 at 7:44 PM, Real Wes <
thereal...@outlook.com>
wrote:
Here is a simple Java client _for enterprise use_ that I
developed
for
Geode and distributed to several enterprises. It has
similarities
and
differences for your goal. This project creates both
server
and
client
regions dynamically. It lists regions, alters regions…
really
anything
that GFSH can do. It differs in that it calls GFSH to do
its
work
rather
than a Java interface. You can think of this as a Java
API
on
top
of
GFSH.
You can use this model and keep the similarities while
adjusting
for
the
Java interface.
Similarities
- Client uses SDG and complies with JSR-107
[the Cache Manager is here:
https://github.com/Pivotal-
Data-Engineering/ClusterManagement/tree/master/
cluster-management-client/src/main/java/com/geode/cache/manager
]
- After the server creates its region, client creates its
region
[ see createRegions at: https://github.com/Pivotal-
Data-Engineering/
ClusterManagement/blob/master/cluster-management-
client/src/main/java/com/geode/cache/manager/
RegionCreator.java<https://
github.com/Pivotal-Data-Engineering/ClusterManagement/
blob/master/cluster-
management-client/src/main/java/com/geode/cache/manager/
RegionCreator.java>
]
Differences
Server dynamically calls GFSH to execute commands
How it works
- Whenever the client calls a region that does not exist,
the
client
looks
for a <region name>.properties file. If the properties
file
exists,
the
region is created with the dynamic properties.
- If <region name>.properties does not exist, default
region
properties
are loaded and used to create the region.
- properties are formed into a GFSH string and passed to
a
server
function
- The server function calls GFSH using internal calls. It
calls
the
GfshParser, executes the command, and then returns the
parsed
results.
[see https://github.com/Pivotal-Data-Engineering/
ClusterManagement/blob/master/
cluster-management-server/src/
main/java/com/geode/gfsh/GfshCommand.java]
Limitations
It uses internal calls to call GFSH. I have made requests
to
make
this
a
stable public interface. Andrew Baker had a good idea to
return
gfsh
results in JSON format with a new —results=json property
to
pass
to
GFSH.
Strengths
That is uses GFSH can be viewed as a strength to keep a
consistent
API,
whether Java or GFSH
TESTS
You can start by running server tests at:
https://github.com/Pivotal-
Data-Engineering/ClusterManagement/tree/master/
cluster-management-server/src/test/java/com/geode/gfsh
Client tests are at:
https://github.com/Pivotal-Data-Engineering/
ClusterManagement/tree/master/
cluster-management-client/src/
test/java/com/geode/management/client
Regards,
Wes Williams
On Apr 24, 2017, at 6:51 PM, Kirk Lund <kl...@apache.org
<mailto:
klund
@apache.org>> wrote:
A couple things I'd like to see:
1) completely new API that doesn't reuse the old API
classes
(or
at
least
not the giant classes such as Cache and Region
interfaces)
2) separation of API and Impl so that users can compile
their
code
against
a dedicated client API jar
On Mon, Apr 24, 2017 at 3:03 PM, Fred Krone <
fkr...@pivotal.io
<mailto:
fkro
n...@pivotal.io>> wrote:
In an effort to improve Java developer ease of use for
caching
with
Geode I
am looking for feedback on going forward with creating a
Java
client.
This
client will allow for server-side region creation and
distributed
data
caching. This would allow for a thin client that fits
with
microservice
caching patterns and also abstracts a cleaner
client-server
experience
driven interface.
Initially we were going to update the Region interface
but
were
concerned
with breaking existing applications. We also would like
to
provide
Region
creation to a client application and so propose here
solving
both
of
these
areas with a Java client.
It would have new project repo for the client.
It would provide new Region interface for clients. The
specifics
of
the
API design are too lengthy for this conversation but
implementation
will
resemble JSR 107 Cache interface.
It would use the new security framework.
*An example*,
The client application simply creates an instance of
client
and
points
it
to the locator:
org.apache.geode.client.Client client =
Client.create(locatorHost,
locatorPort);
Client has the following methods:
package org.apache.geode.client;
public interface GeodeClient {
/**
* creates the region on the servers, or gets the region
if
it
exits,
returns a PROXY region
*/
public Region getOrCreateRegion(RegionAttributes
attributes,
String
name);
/**
* Returns a PROXY region if the region exists on the
server
*/
public Region getRegion(String name);
MVP
The smallest set of features to test this idea, learn and
iterate,
and
get
the client into the communities hands for future
iterations
is:
Create a server side Region from a client
Region interface has CRUD on par with javax.cache.Cache
(from
JSR
107)
Calls are asynchronous -- futures
Also would like feedback on which future functionality
would
be
most
useful
from a thin client:
Function execution
Durable clients
User defined serialization
Register interest
Queries
CQ
Near side caching
Create disk stores from client
Region group membership
Client subscription load balancing
Transactions
Thanks,
-Fred
--
~/William
