The entirety of KIP-14 doesn't need to be scoped for the 1.4 release. JIRAs do need to be filed for the work detailed there and iterative progress will be made across multiple releases.
I think that the initial goal should be to fill the role that the KnoxLine example represented and drove interest in a Knox SQL client. Being able to work with the results of a SQL query is an added benefit of this approach and we can have some basic operations for doing just that. I don't see the overall Knox metadata endpoint making it in the 1.4 release timeframe. I do see some of the underlying capabilities of things like datasources being configurable and used to interact with SQL engines rather than having to provide connection details for each session. I have an early version of this already working locally. On Mon, Nov 11, 2019 at 2:38 PM Kevin Risden <kris...@apache.org> wrote: > Based on that we are ~1/2 way through November and Thanksgiving in the US > around the corner, I don't see much of the KnoxShell pieces getting > integrated before end of November to make an end of November release. > Specifically the following section: > > With the continued increase in cloud based deployments and Knox Gateway use > > in securely accessing the exposed resources, we will concentrate on > > KnoxShell as a first class environment for this access. This will likely > > include an API for discovering metadata about the resources exposed > through > > Knox, the required authentication mechanisms, resource types and public > > certs. It will also include Custom GroovyShell Commands for the KnoxShell > > environment to help interact with the remote clusters and resultsets as > > local in-memory tables. I will be start a KIP to try and articulate this > > vision and related 1.4. usecases as well. > > > > The KIP was just started so would be good to flesh that out more instead of > rushing it into 1.4.0. There are a lot of moving pieces in that paragraph > and would be good to make sure the Jiras are created and scoped > appropriately. > > In addition to what was mentioned as features, there have been multiple new > service definitions (Impala, Kudu, NiFi Registry) added as well as fixes to > existing service definitions (Atlas, Livy, Ranger, Spark, YARN). > > So +1 to an end of November release, but need to make sure not trying to > rush in new things just because a release will happen. There will be more > releases. > > > Kevin Risden > > > On Fri, Nov 1, 2019 at 11:51 AM Sandeep Moré <moresand...@gmail.com> > wrote: > > > Thanks for starting the planning thread Larry ! > > Agree with the theme and the release date for 1.4.0. > > > > +1 > > > > Best, > > Sandeep > > > > On Thu, Oct 31, 2019 at 11:53 AM larry mccay <lmc...@apache.org> wrote: > > > >> Folks - > >> > >> Out last release with end of July, I apologize for the delay in starting > >> the planning thread for 1.4. > >> > >> We currently have a backlog of ~65 JIRAs slated for a Fix Version of > 1.4. > >> > >> There has been some work going on within KnoxShell to provide a general > >> purpose representation for tabular data. This will be leveraged for > >> rendering SQL query results as well as CSV files and simple processing > >> within KnoxShell. I will be writing up a KIP to represent the overall > >> vision for this work and initial set of usecases. > >> > >> We also have Cloudera Manager based discovery emerging and we should > >> target an initial set of services to enable for CM/CDH and CDP > deployments > >> where CM is available. > >> > >> With the continued increase in cloud based deployments and Knox Gateway > >> use in securely accessing the exposed resources, we will concentrate on > >> KnoxShell as a first class environment for this access. This will likely > >> include an API for discovering metadata about the resources exposed > through > >> Knox, the required authentication mechanisms, resource types and public > >> certs. It will also include Custom GroovyShell Commands for the > KnoxShell > >> environment to help interact with the remote clusters and resultsets as > >> local in-memory tables. I will be start a KIP to try and articulate this > >> vision and related 1.4. usecases as well. > >> > >> I propose that the CM based Service Discovery and KnoxShell access to > >> remote clusters be the primary themes of the Apache Knox 1.4 release. > >> > >> I also propose that we target the end of November as the release date > for > >> 1.4. > >> > >> Thoughts? > >> > >> --larry > >> > > >
