Wes, Those are almost all administrative commands and have no place on the client API. They belong on an administrative API or as I'm arguing a series of MBeans/JMX as it is already an established standard.
-Jake On Wed, Apr 26, 2017 at 8:09 AM Wes Williams <wwilli...@pivotal.io> wrote: > Now we're getting some precision. Let's talk about the "raw" Geode > proprietary bad ass API! Would that "raw" Geode proprietary bad ass API > "raw" > Geode proprietary bad ass API that we're talking about be centered around > the commands found here: > > https://github.com/apache/geode/tree/rel/v1.1.1/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands > > Or somewhere else? > > *Wes Williams | Pivotal Advisory **Data Engineer* > 781.606.0325 > http://pivotal.io/big-data/pivotal-gemfire > > On Tue, Apr 25, 2017 at 11:41 PM, Jacob Barrett <jbarr...@pivotal.io> > wrote: > > > Java and its community have standards for all of these issue so why > > re-invent the wheel. The market doesn't want proprietary anymore, they > want > > standards and mobility. > > > > Configuration of the server should happen through MBeans. You can wrap > that > > in gfsh for command line, REST for remote web based admin, use JConsole > or > > any other number of JMX based enterprise management tools. By using > MBeans > > the server can easily expose new discovered services without the need to > > code specific gfsh commands, REST interfaces or APIs. There is no reason > my > > SDG can't be retooled to "discover" the configuration from these MBeans > as > > well rather than having to be touched every time we add or change > > something. There are tools and books already written that implementors > can > > consult on MBeans. There isn't anything out there on gfsh commands. > > > > If we want to play in the Java community, especially J2EE (the other 50% > of > > Java that isn't Spring), then we had better have a JSR-107 answer no > matter > > what the pain is to provide it. I can pull dozens of books of the shelf > > that teach me how to effectively use a JCache, how many can I pull off > the > > shelf that teach me Geode's API? How many engineers can I get > applications > > form by saying "must have Geode API knowledge"? I can find people with > > JCache knowledge though. So from in implementor's perspective having > > standards is a must. Now I don't think the JSR-107 interface should be > the > > root interface but rather a facade on the "raw" Geode proprietary bad ass > > API. That API should be 100% asynchronous (reactive, SEDA, whatever the > > current buzzword for asynchronous is today). Around that API we can > provide > > facades for JSR 107, ConcurrentMap (our current yet not so well behaving > > API), List, Queue, etc. Maybe even JPA, JCA, etc. The thought of putting > > all those features into a single API makes my head exploded and they > don't > > need to be like they are today. > > > > > > > > On Tue, Apr 25, 2017 at 8:25 PM Wes Williams <wwilli...@pivotal.io> > wrote: > > > > > A couple of points to interact with John's points. > > > > > > GFSH as API > > > > > > We agree that GFSH is a DSL, and a really good and useful one. We agree > > > that we don't want our API tied to GFSH syntax. I view the valuable > part > > of > > > GemFire admin as the Java code underneath GFSH, or the "Commands." > > > > > > For example, to create a JAVA API to "Create Region", why not create a > > > Java API around CreateAlterDestroyRegionCommands? In this way, GFSH and > > the > > > JAVA API share the same code. It seems too obvious yet I don't see it > > being > > > recommended. Why not? (Note: the hard-coded formatting would need to > be > > > removed). > > > > > > Once you have the Java/GFSH/REST API as common code, you then refactor > > it. > > > What's the objection to this approach? Once you open Java API's to do > > > everything that GFSH does, then you have now unshackled the power of > the > > > developer (the JAVA API) and the admin (GFSH API). > > > > > > > > > REST API > > > > > > I've found that most don't want to use the Dev REST API because it's > > > attached to a server rather than the cluster. HA? > > > > > > > > > *Wes Williams | Pivotal Advisory **Data Engineer* > > > 781.606.0325 <(781)%20606-0325> > > > http://pivotal.io/big-data/pivotal-gemfire > > > > > > On Tue, Apr 25, 2017 at 7:01 PM, Fred Krone <fkr...@pivotal.io> wrote: > > > > > > > Good feedback. > > > > > > > > This would use the new protocol. I should have mentioned that. > > > > > > > > The original driver for this was the Region API needs either an > update > > or > > > > an alternative. Updating has a few drawbacks: Region wasn't designed > > > with > > > > open source in-mind and as Swap mentioned it is naturally tightly > > > coupled. > > > > Members of the community are already working to update Region but > > > something > > > > gets fixed and it breaks something for someone else. I think it's > much > > > > better to provide a new interface that implements the first part of > JSR > > > 107 > > > > (javax.cache) and get the ball rolling for the community and, > perhaps, > > > over > > > > time deprecate Region (although that's not a primary objective). > > > > > > > > A Java driver will probably get built regardless just to give the new > > > > protocol some legs. That driver also needs a decent caching > interface. > > > JSR > > > > 107 has already solved that part. So let's get started on it. If > the > > > > community wants to go the whole way and continue JSR 107 > implementation > > > > after that that's awesome. Functions can also be added, etc. > > > > > > > > I intentionally did not mention anything about PCF as this pertains > to > > > > Geode itself as an open source offering and developer experience. > I'm > > > > writing as a member of the community. Ie: I'm a developer who would > > like > > > to > > > > add some caching to my application -- I can download either Geode or > > > > Hazelcast for free and right now it's a no brainer. Not that we > > wouldn't > > > > keep PCF in-mind but it's out of scope for this thread. I do believe > > > > getting started on a Java driver for the protocol and a standardized > > > > caching API are easily leveraged wins across the board. > > > > > > > > > > > > > > > > > > > > On Tue, Apr 25, 2017 at 3:20 PM, Swapnil Bawaskar < > > sbawas...@pivotal.io> > > > > wrote: > > > > > > > > > I had looked at the JCache in the past and here are some of the > > things > > > I > > > > > noted: > > > > > > > > > > Naming convention: Geode's region is a Cache in JSR-107, and > Geode's > > > > Cache > > > > > is a CacheManager. I think it would be too confusing to change the > > > > meaning > > > > > of cache. Also, how do you document this given that Cache means > > > different > > > > > things if you are talking JCache vs Geode. > > > > > > > > > > The way to create a Cache in JSR-107 is: > > > > > Cache<Integer, Date> cache = manager.createCache(cacheName, > > > Configuration > > > > > c); > > > > > Where it is upto the implementation to extend Configuration. Given > > > this, > > > > > users will not be able to switch from an existing implementation to > > > ours; > > > > > will have to write new code specially Configuration, making > callbacks > > > > > serializable etc. > > > > > > > > > > JSR-107 will not be limited to the client. Server side callbacks > like > > > > > CacheLoader, CacheListener etc. will need handle on jsr-107 > “cache”. > > > > > > > > > > JSR-107 supports features like an EntryProcessor, which is a > function > > > > > invoked atomically on an entry operation. We will have to make > > invasive > > > > > changes to Geode to support this. > > > > > > > > > > Given these, I don't think supporting JSR-107 is trivial. > > > > > > > > > > On Tue, Apr 25, 2017 at 2:55 PM Dan Smith <dsm...@pivotal.io> > wrote: > > > > > > > > > > > What transport are you planning on using? REST, or the current > > binary > > > > > > protocol? Or is this just a wrapper around the existing java > client > > > > APIs? > > > > > > > > > > > > If this about creating a new API, I agree with what John is > saying > > > that > > > > > we > > > > > > need reduce the number of APIs we have to do the same thing in > > java. > > > It > > > > > > seems especially confusing if we end up with different APIs that > > > > support > > > > > > distinct features - like being able to create a region on the > > server > > > > with > > > > > > this new API but only being able to invoke a function with the > old > > > API. > > > > > > > > > > > > The other thing I think we need to avoid is being able to do > things > > > > from > > > > > > the client (or gfsh, or xml, ...) that don't have a java API on > the > > > > > server. > > > > > > I'm assuming your getOrCreateRegion region is supposed behave > like > > > the > > > > > gfsh > > > > > > command and update the cluster configuration? +++1 to Wes's > > > suggestion > > > > > have > > > > > > a public API for executing these gfsh operations. > > > > > > > > > > > > I really think we need to work on having *one* authoritative > public > > > API > > > > > > that contains everything that we support on the server. The > > protocols > > > > we > > > > > > support (REST, binary, ...) and the client drivers that use those > > > > > protocols > > > > > > should just be ways of accessing that API. The Java API on the > > server > > > > > right > > > > > > now the closest thing we have to this, but there are things you > can > > > do > > > > > with > > > > > > gfsh and things you can do with the current client that have no > > java > > > > API > > > > > > equivalent. I think we really need to fix that! > > > > > > > > > > > > Also, preferably we won't have to hand code a bunch of stuff in > > every > > > > > > client driver and protocol whenever we add a new feature. For > > example > > > > if > > > > > > were to add a geode function to invoke each new API we add, that > > new > > > > API > > > > > > would be accessible from REST, gfsh, the java client etc. Later > we > > > > could > > > > > > add syntatic sugar to make the new API prettier, but if we had a > > low > > > > > level > > > > > > API that automatically exposed all new features that would make > > > adding > > > > > new > > > > > > features much less painful. > > > > > > > > > > > > I do like the idea of adding a reactive API. > > > > > > > > > > > > Supporting JSR-107 might be nice - but that could probably be > > > written > > > > > just > > > > > > as a wrapper around the current API without too much work. I > don't > > > > think > > > > > we > > > > > > should do anything for JSR-107 other than provide a JSR-107 > > > compatible > > > > > > wrapper - if someone wants additional geode specific features > they > > > > should > > > > > > drop into the existing API. > > > > > > > > > > > > I also like the idea of a smaller client jar and dependencies, > but > > I > > > > > think > > > > > > there is a huge amount of room for improvement in our existing > > client > > > > > just > > > > > > by refactoring the code a bit more and shinking geode-core into a > > > bare > > > > > > minimum. > > > > > > > > > > > > -Dan > > > > > > > > > > > > On Mon, Apr 24, 2017 at 8:20 PM, Fred Krone <fkr...@pivotal.io> > > > wrote: > > > > > > > > > > > > > That's great, Wes. I will take a look. Thanks. > > > > > > > > > > > > > > John -- good feedback to consider and I was hoping this would > > come > > > > up. > > > > > > > > > > > > > > "In my mind, there really are only 2 approaches... *Spring* and > > > > > > > non-*Spring*, > > > > > > > or rather PCF and non-PCF, and since PCF is primarily based on > > Boot > > > > > > (given > > > > > > > Microservices/applications are the new concurrency), then I am > > > really > > > > > > > saying the same thing." > > > > > > > > > > > > > > This would be cover non spring and attempt to give the > community > > > > > > something > > > > > > > that had the same standardized experience as JSR 107 -- > starting > > > with > > > > > the > > > > > > > Cache interface itself. Although we don't necessarily have to > > > > > implement > > > > > > > Cache, it's method signatures are essentially a (non spring) > > > caching > > > > > > > standard for Java developers at this point. We considered > full > > > > blown > > > > > > JSR > > > > > > > 107 implementation but thought it was too robust for the needs > > > > > mentioned > > > > > > > (that's not to say we couldn't get there incrementally). I > think > > > > those > > > > > > > needs still exist open-source outside of PCF and don't > > cannibalize. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Apr 24, 2017 at 7:44 PM, Real Wes < > > thereal...@outlook.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Here is a simple Java client _for enterprise use_ that I > > > developed > > > > > for > > > > > > > > Geode and distributed to several enterprises. It has > > similarities > > > > and > > > > > > > > differences for your goal. This project creates both server > and > > > > > client > > > > > > > > regions dynamically. It lists regions, alters regions… > really > > > > > anything > > > > > > > > that GFSH can do. It differs in that it calls GFSH to do its > > work > > > > > > rather > > > > > > > > than a Java interface. You can think of this as a Java API on > > top > > > > of > > > > > > > GFSH. > > > > > > > > > > > > > > > > You can use this model and keep the similarities while > > adjusting > > > > for > > > > > > the > > > > > > > > Java interface. > > > > > > > > > > > > > > > > Similarities > > > > > > > > - Client uses SDG and complies with JSR-107 > > > > > > > > [the Cache Manager is here: https://github.com/Pivotal- > > > > > > > > Data-Engineering/ClusterManagement/tree/master/ > > > > > > > > > cluster-management-client/src/main/java/com/geode/cache/manager > > ] > > > > > > > > - After the server creates its region, client creates its > > region > > > > > > > > [ see createRegions at: https://github.com/Pivotal- > > > > > > > Data-Engineering/ > > > > > > > > ClusterManagement/blob/master/cluster-management- > > > > > > > > > > > > > > client/src/main/java/com/geode/cache/manager/ > > > > RegionCreator.java<https:// > > > > > > > > github.com/Pivotal-Data-Engineering/ClusterManagement/ > > > > > > > blob/master/cluster- > > > > > > > > management-client/src/main/java/com/geode/cache/manager/ > > > > > > > RegionCreator.java> > > > > > > > > ] > > > > > > > > > > > > > > > > Differences > > > > > > > > Server dynamically calls GFSH to execute commands > > > > > > > > > > > > > > > > How it works > > > > > > > > - Whenever the client calls a region that does not exist, the > > > > client > > > > > > > looks > > > > > > > > for a <region name>.properties file. If the properties file > > > exists, > > > > > the > > > > > > > > region is created with the dynamic properties. > > > > > > > > - If <region name>.properties does not exist, default region > > > > > properties > > > > > > > > are loaded and used to create the region. > > > > > > > > - properties are formed into a GFSH string and passed to a > > server > > > > > > > function > > > > > > > > - The server function calls GFSH using internal calls. It > calls > > > the > > > > > > > > GfshParser, executes the command, and then returns the parsed > > > > > results. > > > > > > > > [see https://github.com/Pivotal-Data-Engineering/ > > > > > > > > ClusterManagement/blob/master/cluster-management-server/src/ > > > > > > > > main/java/com/geode/gfsh/GfshCommand.java] > > > > > > > > > > > > > > > > Limitations > > > > > > > > It uses internal calls to call GFSH. I have made requests to > > make > > > > > this > > > > > > a > > > > > > > > stable public interface. Andrew Baker had a good idea to > return > > > > gfsh > > > > > > > > results in JSON format with a new —results=json property to > > pass > > > to > > > > > > GFSH. > > > > > > > > > > > > > > > > Strengths > > > > > > > > That is uses GFSH can be viewed as a strength to keep a > > > consistent > > > > > API, > > > > > > > > whether Java or GFSH > > > > > > > > > > > > > > > > TESTS > > > > > > > > You can start by running server tests at: > > > > > https://github.com/Pivotal- > > > > > > > > Data-Engineering/ClusterManagement/tree/master/ > > > > > > > > cluster-management-server/src/test/java/com/geode/gfsh > > > > > > > > > > > > > > > > Client tests are at: > > > https://github.com/Pivotal-Data-Engineering/ > > > > > > > > ClusterManagement/tree/master/cluster-management-client/src/ > > > > > > > > test/java/com/geode/management/client > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > Wes Williams > > > > > > > > > > > > > > > > On Apr 24, 2017, at 6:51 PM, Kirk Lund <kl...@apache.org > > <mailto: > > > > > klund > > > > > > > > @apache.org>> wrote: > > > > > > > > > > > > > > > > A couple things I'd like to see: > > > > > > > > > > > > > > > > 1) completely new API that doesn't reuse the old API classes > > (or > > > at > > > > > > least > > > > > > > > not the giant classes such as Cache and Region interfaces) > > > > > > > > 2) separation of API and Impl so that users can compile their > > > code > > > > > > > against > > > > > > > > a dedicated client API jar > > > > > > > > > > > > > > > > On Mon, Apr 24, 2017 at 3:03 PM, Fred Krone < > fkr...@pivotal.io > > > > > <mailto: > > > > > > > fkro > > > > > > > > n...@pivotal.io>> wrote: > > > > > > > > > > > > > > > > In an effort to improve Java developer ease of use for > caching > > > with > > > > > > > Geode I > > > > > > > > am looking for feedback on going forward with creating a Java > > > > client. > > > > > > > This > > > > > > > > client will allow for server-side region creation and > > distributed > > > > > data > > > > > > > > caching. This would allow for a thin client that fits with > > > > > > microservice > > > > > > > > caching patterns and also abstracts a cleaner client-server > > > > > experience > > > > > > > > driven interface. > > > > > > > > > > > > > > > > Initially we were going to update the Region interface but > were > > > > > > concerned > > > > > > > > with breaking existing applications. We also would like to > > > provide > > > > > > > Region > > > > > > > > creation to a client application and so propose here solving > > both > > > > of > > > > > > > these > > > > > > > > areas with a Java client. > > > > > > > > > > > > > > > > It would have new project repo for the client. > > > > > > > > > > > > > > > > It would provide new Region interface for clients. The > > specifics > > > > of > > > > > > the > > > > > > > > API design are too lengthy for this conversation but > > > implementation > > > > > > will > > > > > > > > resemble JSR 107 Cache interface. > > > > > > > > > > > > > > > > It would use the new security framework. > > > > > > > > > > > > > > > > > > > > > > > > *An example*, > > > > > > > > > > > > > > > > The client application simply creates an instance of client > and > > > > > points > > > > > > it > > > > > > > > to the locator: > > > > > > > > > > > > > > > > > > > > > > > > org.apache.geode.client.Client client = > > > Client.create(locatorHost, > > > > > > > > locatorPort); > > > > > > > > > > > > > > > > > > > > > > > > Client has the following methods: > > > > > > > > > > > > > > > > package org.apache.geode.client; > > > > > > > > > > > > > > > > > > > > > > > > public interface GeodeClient { > > > > > > > > > > > > > > > > /** > > > > > > > > > > > > > > > > * creates the region on the servers, or gets the region if > it > > > > > exits, > > > > > > > > returns a PROXY region > > > > > > > > > > > > > > > > */ > > > > > > > > > > > > > > > > public Region getOrCreateRegion(RegionAttributes attributes, > > > > String > > > > > > > > name); > > > > > > > > > > > > > > > > > > > > > > > > /** > > > > > > > > > > > > > > > > * Returns a PROXY region if the region exists on the server > > > > > > > > > > > > > > > > */ > > > > > > > > > > > > > > > > public Region getRegion(String name); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > MVP > > > > > > > > > > > > > > > > The smallest set of features to test this idea, learn and > > > iterate, > > > > > and > > > > > > > get > > > > > > > > the client into the communities hands for future iterations > is: > > > > > > > > > > > > > > > > > > > > > > > > Create a server side Region from a client > > > > > > > > > > > > > > > > Region interface has CRUD on par with javax.cache.Cache (from > > JSR > > > > > 107) > > > > > > > > > > > > > > > > Calls are asynchronous -- futures > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Also would like feedback on which future functionality would > be > > > > most > > > > > > > useful > > > > > > > > from a thin client: > > > > > > > > > > > > > > > > Function execution > > > > > > > > > > > > > > > > Durable clients > > > > > > > > > > > > > > > > User defined serialization > > > > > > > > > > > > > > > > Register interest > > > > > > > > > > > > > > > > Queries > > > > > > > > > > > > > > > > CQ > > > > > > > > > > > > > > > > Near side caching > > > > > > > > > > > > > > > > Create disk stores from client > > > > > > > > > > > > > > > > Region group membership > > > > > > > > > > > > > > > > Client subscription load balancing > > > > > > > > Transactions > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > -Fred > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
