Hi Nick, Perhaps my choice of the word "infrastructure" is rather poor -- overstating (even *melodramatizing*!) what is there.
I'm hopeful that when you take a look at it, that the structures and the reasons that the structures are there will be apparent. (If not, then my README text failed one of its initial tests, and needs to be reworked to provide further clarifications.) But, in short, everything that is there, I found necessary to be there in order to (1) facilitate proper automatic testing of *all* components of an archetype, and (2) provide automation of archetype creation steps that would otherwise have to manually be done by a contributor and manually verified by a committer. (The folks at the Maven project provide some wonderful archetype plugin tools to accomplish a lot of this, and these are being leveraged to the hilt; but a few details they don't seem to provide for had to be provided via simple bash scripts and xslt transformation.) At this point, I'll let you take a look at things before I attempt to elaborate further... Thanks, Dan On Fri, Jan 29, 2016 at 2:06 AM, Nick Dimiduk <ndimi...@gmail.com> wrote: > Hi Daniel, > > Thanks again for picking this one up and really working it! I'm not clear > on what new infrastructure is needed. The archetypes would be added as new > modules to our existing maven structure, built when existing code is built, > tests run when existing code tests are run, and deployed the same. Is that > not so? Let me spend some time with your patches (I know I keep saying > that...) and come back for a more coherent conversation. > > -n > > On Wednesday, January 27, 2016, Daniel Vimont <dan...@commonvox.org> > wrote: > > > One point to add to this discussion: the intention is to backport this to > > make it available in an upcoming 1.x minor release, so that new > archetypes > > are publicly available via the Maven central repository as soon as > > possible. > > > > As Sean brings up over in the HBASE-14877 JIRA entry > > <https://issues.apache.org/jira/browse/HBASE-14877>, we need to > determine > > "what level of 'done' we're looking for before we do that". > > > > While I imagine that this will not be anywhere near as involved as the > > backporting of the new hbase-spark stuff to branch-1 (see HBASE-14160 > > <https://issues.apache.org/jira/browse/HBASE-14160>), this patch does > > introduce *new infrastructure* for developing and maintaining new > artifacts > > (in this case, Maven archetypes), so it certainly does merit some review > > before we open up a new JIRA and start adding this functionality to > > branch-1. > > > > On Wed, Jan 27, 2016 at 4:07 PM, Daniel Vimont <dan...@commonvox.org > > <javascript:;>> wrote: > > > > > Work has been underway for the last month on task HBASE-14877 > > > <https://issues.apache.org/jira/browse/HBASE-14877>, the first subtask > > of > > > HBASE-14876 (in Nick's original wording from that JIRA entry: "To help > > > onboard new users, we should *provide maven archetypes for hbase client > > > applications*"). > > > > > > The three originally proposed *Maven archetypes* in HBASE-14876 were > > > split into three subtasks, for: > > > > > > 1. an hbase-client dependent application, > > > 2. an hbase-shaded-client dependent application, and > > > 3. a map-reduce job example with hbase-server dependency. > > > > > > *(Note that a fourth subtask has since been added -- for an hbase-spark > > > oriented archetype.)* > > > > > > *The first subtask (HBASE-14877) provides:* > > > > > > *[a] a new infrastructure* for creation and maintenance of > HBase-related > > > Maven archetypes, and > > > [*b] the first archetype* from the list above (to be used by end-users > to > > > autogenerate a fully-configured Maven project with hbase-client > > dependency > > > and fully-functional sample Java code). > > > > > > > > > *It is item [a] above -- a new infrastructure for creation/maintenance > of > > > archetypes -- that we would particularly like for > contributors/committers > > > to take a look at, and to offer up comments, questions, suggestions, > > etc.* > > > The README text in the new hbase-archetypes subdirectory is intended to > > be > > > a brief yet comprehensive introduction to how this new infrastructure > > > works, and how a contributor would add new Maven archetypes in the > > future. *(It > > > is intended that it be very simple to contribute new archetypes, > without > > > the need for any special expertise with regard to Maven archetypes in > > > general.)* > > > > > > For ease of access, I'll just paste in the README text below. *(Note > that > > > this text will soon be reformatted into markdown format, but this is > the > > > intended content.)* > > > > > > Besides reviewing this text, you can earn extra gratitude-points if > you: > > > > > > - > > > *set up a sandbox environment, * > > > - > > > *install the latest patch (HBASE-14877-v4.patch > > > < > > > https://issues.apache.org/jira/secure/attachment/12784323/HBASE-14877-v4.patch > > >) > > > against the HBase master branch, * > > > - > > > *follow the instructions at the end of the README (below) to generate a > > > new project from the archetype (emulating the experience of an > > end-user), > > > and * > > > - *review the results of that process, as well!* > > > > > > THANKS VERY MUCH, > > > Dan Vimont > > > > > > > > > > > > > > > ------------------------------------------------------------------------------- > > > *README.txt* > > > > > > > > > ------------------------------------------------------------------------------- > > > *The hbase-archetypes subproject of hbase provides an infrastructure > for* > > > *creation and maintenance of Maven archetypes[1] pertinent to HBase. > > Upon* > > > *deployment to the archetype catalog[2] of the central Maven > > > repository[3],* > > > *these archetypes may be used by end-user developers to autogenerate > > > completely* > > > *configured Maven projects (including fully-functioning sample code) > > > through* > > > *invocation of the archetype:generate goal of the > > > maven-archetype-plugin[4].* > > > > > > *Notes for contributors and committers to the HBase project* > > > *==========================================================* > > > *The hbase-archetypes project contains a separate subproject for each > > > archetype.* > > > *The top level components of such a subproject comprise a complete, > > > standalone* > > > *exemplar Maven project containing (a) a src directory with sample, > > fully-* > > > *functioning code in the ./main and ./test subdirectories, (b) a > pom.xml > > > file* > > > *defining all required dependencies, and (c) any additional resources > > > required* > > > *by the exemplar project. (For example, the components of the > > > hbase-client-* > > > *project consist of (a) sample code ./src/main/.../HelloHBase.java and* > > > *./src/test/.../TestHelloHBase.java, (b) a pom.xml file establishing > > > dependency* > > > *upon hbase-client and test-scope dependency upon hbase-testing-util, > and > > > (c) a* > > > *log4j.properties resource file.)* > > > > > > *** Overview of the creation of archetypes during the hbase install > > > process: *** > > > *During the mvn install process, all standalone exemplar projects in > the* > > > *hbase-archetypes subdirectory are first packaged/tested/installed, and > > > then the* > > > *following steps are executed in > hbase-archetypes/hbase-archetype-builder > > > (via* > > > *the pom.xml, bash scripts, and xsl templates in that subdirectory):* > > > * (1) For each exemplar project, resources are copied (via > > > maven-resources-* > > > *plugin) and transformed (via xml-maven-plugin xslt functionality) to > > the* > > > *exemplar project's ./target/build-archetype subdirectory[5].* > > > * (2) The script createArchetypes.sh is executed to invoke the > > > maven-archetype-* > > > *plugin's create-from-project goal within each exemplar project's* > > > *./target/build-archetype subdirectory. For each exemplar project, this > > > creates* > > > *a corresponding Maven archetype in the ./target/build-* > > > *archetype/target/generate-sources/archetype subdirectory. (Note that > > this > > > step* > > > *always issues two platform-encoding warnings per archetype, due to > > > hard-wired* > > > *behavior of the maven-archetype-plugin[6].)* > > > * (3) The pom.xml file of each newly-created archetype is copied (via > > > maven-* > > > *resources-plugin) and transformed (via xml-maven-plugin xslt > > > functionality)[7].* > > > * (4) The script installArchetypes.sh is executed to install each > > > archetype* > > > *into the local Maven repository, ready for deployment to the central > > > Maven* > > > *repository. (Note that installation of an archetype automatically > > > includes* > > > *invocation of integration-testing prior to install, which performs a > > test* > > > *generation of a project from the archetype.)* > > > > > > *** To add a new archetype to the hbase-archetypes collection: *** > > > * (1) Create a new subdirectory in hbase-archetypes, populated with a* > > > *completely configured Maven project, which will serve as the exemplar > > > project* > > > *of the new archetype. (It may be most straightforward to simply copy > the > > > src* > > > *and pom.xml components from one of the existing exemplar projects, > > > replace the* > > > *src/main and src/test code, and modify the pom.xml file's > > <dependencies>,* > > > *<artifactId>, <name>, and <description> elements.)* > > > * (2) Modify the hbase-archetype-builder/pom.xml file: (a) add the new > > > exemplar* > > > *project to the <modules> element, and (b) add appropriate <execution> > > > elements* > > > *and <transformationSet> elements within the <plugin> elements (using > > the* > > > *existing entries from already-existing exemplar projects as a guide).* > > > * (3) Add appropriate entries for the new exemplar project to the* > > > *createArchetypes.sh and installArchetypes.sh scripts in the > > > hbase-archetype-* > > > *builder subdirectory (using the existing entries as a guide).* > > > > > > *** For additional testing/inspection of an archetype in this > collection: > > > *** > > > *Although integration-testing (which is automatically performed for > each* > > > *archetype during the install process) already performs test generation > > of > > > a* > > > *project from an archetype, it may often be advisable to do further > > manual* > > > *testing of a newly built and installed archetype, particularly to > > examine > > > and* > > > *test a project generated from the archetype (mimicking the end-user > > > experience* > > > *of utilizing the archetype). Upon completion of the install process > > > outlined* > > > *above, all archetypes will have been installed in the local Maven > > > repository* > > > *and can be tested locally by executing the following:* > > > * mvn archetype:generate -DarchetypeCatalog=local* > > > *This displays a numbered list of all locally-installed archetypes for > > the > > > user* > > > *to choose from for generation of a new Maven project.* > > > *=================================================================* > > > *END of notes for contributors and committers to the HBase project* > > > > > > *Footnotes:* > > > *===========================================* > > > *[1] http://maven.apache.org/archetype/index.html > > > <http://maven.apache.org/archetype/index.html>* > > > *[2] http://repo1.maven.org/maven2/archetype-catalog.xml > > > <http://repo1.maven.org/maven2/archetype-catalog.xml>* > > > *[3] http://search.maven.org/ <http://search.maven.org/>* > > > *[4] > > > > > > http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html > > > < > > > http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html > > >* > > > *[5] Prior to archetype creation, each exemplar project's pom.xml is > > > transformed* > > > * as follows to make it into a standalone project: resource > filtering* > > > * replaces ${project.version} with the literal value of the current* > > > * project.version; and xslt tranformation (a) copies <groupId> and > > > <version>* > > > * subelements of <parent> to make them child elements of the root > > > element,* > > > * and (b) removes the <parent> and <description> elements.* > > > *[6] For an explanation of the platform-encoding warning issued during* > > > * maven-archetype-plugin processing, see the first answer to this* > > > * stackoverflow posting: > http://stackoverflow.com/a/24161287/4112172 > > > <http://stackoverflow.com/a/24161287/4112172>* > > > *[7] Prior to archetype installation, each archetype's pom.xml is > > > transformed* > > > * as follows: a <project.build.sourceEncoding> subelement with > value* > > > * 'UTF-8' is added to the <properties> element. This prevents* > > > * platform-encoding warnings from being issued when an end-user > > > generates* > > > * a project from the archetype.* > > > > > > > > > ------------------------------------------------------------------------------- > > > > > >
