Thanks Jay. There are already some comments from YoungWoo Kim which are taken care of in that PR.
On Tue, Feb 23, 2016 at 6:05 PM, Jay Vyas <[email protected]> wrote: > Thanks again for this , don't apologize for the email. > > We actually are excited about apex, thanks for this! We will get it > reviewed within the next couple of days I'm sure (I'll look some time after > work if nobody else gets to it first). > > > On Feb 23, 2016, at 6:09 AM, Chinmay Kolhatkar <[email protected]> > wrote: > > > > Sorry for another mail. Just want to mention how I tested the patch > before > > creating PR: > > > > Apex package was built using with following docker images: > > 1) Ubuntu trusty: > > docker run --rm -v `pwd`:/ws --workdir /ws > bigtop/slaves:trunk-ubuntu-14.04 > > bash -l -c './gradlew allclean ; ./gradlew apex-pkg' > > > > 2) CentOS 7 > > docker run --rm -v `pwd`:/ws --workdir /ws bigtop/slaves:trunk-centos-7 > > bash -l -c './gradlew allclean ; ./gradlew apex-pkg' > > > > 3) CentOS 6 > > docker run --rm -v `pwd`:/ws --workdir /ws bigtop/slaves:trunk-centos-6 > > bash -l -c './gradlew allclean ; ./gradlew apex-pkg' > > > > 4) OpenSUSE 13.2 > > docker run --rm -v `pwd`:/ws --workdir /ws > > bigtop/slaves:trunk-opensuse-13.2 -l -c './gradlew allclean ; ./gradlew > > apex-pkg' > > > > > > Apex package was installed on following docker containers and verified > for > > sanity: > > 1) bigtop/slaves:trunk-ubuntu-14.04 > > 2) bigtop/slaves:trunk-centos-6 > > 3) bigtop/slaves:trunk-centos-7 > > 4) bigtop/slaves:trunk-opensuse-13.2 > > > > When you have some time, please provide the feedback on the pull request > ( > > https://github.com/apache/bigtop/pull/92) > > > > Thank you, > > Chinmay. > > > > > > > > > > On Mon, Feb 22, 2016 at 9:43 PM, Chinmay Kolhatkar <[email protected]> > > wrote: > > > >> Hello Bigtop Community, > >> > >> I've created a pull request for packing of Apache Apex into bigtop that > >> I'd like to get reviewd and committed to Bigtop repository: > >> https://github.com/apache/bigtop/pull/92 > >> > >> The pull request contains code for packing of apex for both deb and rpm > >> packing. > >> This does not include smoke tests which will be the very next steps once > >> this gets accepted. > >> > >> If you have some time, could you please review this pull request and > >> provide feedback so that we can get these merged in? > >> > >> Also, there is a question related to what should be included in package > >> with respect to packing of runtime dependent jars. > >> Currently, I've included license files of non-ASF based licenses in > >> package. > >> Is there any specific guidelines from bigtop that integrating projects > >> needs to follow on that front? > >> > >> In the meantime, I would like to get some feedback on pull request for > if > >> there are any code change required. > >> > >> Thank you in advance!! > >> > >> Chinmay. > >> > >> > >> > >> On Wed, Feb 17, 2016 at 11:10 AM, Chinmay Kolhatkar <[email protected] > > > >> wrote: > >> > >>> Cos, > >>> Thank for a quick reply. We (apex community) has decided to follow your > >>> recommendation i.e. drop "-incubating". > >>> > >>> Roman, > >>> Apologies, I did not understand your last question earlier. > >>> Yes, I can certainly provide the prototype for rpm+deb packing of Apex > >>> with bigtop. > >>> Here is the link to GH where development is WIP: > >>> https://github.com/chinmaykolhatkar/bigtop/tree/BIGTOP-2314 > >>> > >>> Thanks, > >>> Chinmay. > >>> > >>> > >>> On Tue, Feb 16, 2016 at 9:15 PM, Konstantin Boudnik <[email protected]> > >>> wrote: > >>> > >>>> Dude to known limitations of RPMs "-" aren't allowed for sure. Deb > >>>> doesn't > >>>> have such idiosyncrasy, but there are ppl who are using RPM, which is > >>>> beyond > >>>> me. Anyway... > >>>> > >>>> This is a long standing issue for us as well, as we have a few > incubating > >>>> projects in the stack. My recommendation would be to drop > "-incubating" > >>>> part > >>>> from the package version. Check out how it is done for zeppelin. > >>>> > >>>> Cos > >>>> > >>>>> On Tue, Feb 16, 2016 at 09:04PM, Chinmay Kolhatkar wrote: > >>>>> Hello Everyone, > >>>>> > >>>>> I'm facing an issue while setting base version for rpm package. > >>>>> We want to have version of Apache Apex as "3.3.0-incubating". > >>>>> But when I try to set version in bigtop.bom file, I get error while > >>>>> generating rpm package as rpm does not allow hyphen ('-') in version. > >>>>> > >>>>> I also tried to set 'incubating' as a value as release: version { > base > >>>> = > >>>>> '3.3.0'; pkg = base; release = 'incubating' } > >>>>> But that doesn't seem to take any effect. > >>>>> > >>>>> Can you please suggest how can we achieve this? > >>>>> > >>>>> Thanks, > >>>>> Chinmay. > >>>>> > >>>>> > >>>>> On Mon, Feb 15, 2016 at 11:38 AM, Chinmay Kolhatkar < > >>>> [email protected]> > >>>>> wrote: > >>>>> > >>>>>> Hi Roman, > >>>>>> > >>>>>> Thanks for the feedback. > >>>>>> Here are the answers to the question in the same order: > >>>>>> 1. Apex is a YARN-native platform. Hence apex needs YARN to be > there. > >>>>>> 2. conf/ directory recently got removed. This is not needed anymore > >>>> in the > >>>>>> package. > >>>>>> 3. Here is the link to GH repo for apex platform: > >>>>>> https://github.com/apache/incubator-apex-core > >>>>>> The source as of now does not output any package. do-component > >>>> build > >>>>>> would compile the apex platform and generate a rpm/deb package using > >>>> files > >>>>>> present in source code. Hence there is no prototype of the package. > >>>>>> Is that the prototype you're looking for? > >>>>>> > >>>>>> Thanks, > >>>>>> Chinmay. > >>>>>> > >>>>>> > >>>>>> On Mon, Feb 15, 2016 at 11:11 AM, Roman Shaposhnik < > >>>> [email protected]> > >>>>>> wrote: > >>>>>> > >>>>>>> Looks good to me. Three questions: > >>>>>>> 1. Does apex provide standalone daemon or does only run on YARN ? > >>>>>>> 2. What's the intended use for content under /usr/lib/apex/conf ? > >>>>>>> 3. Any chance you can provide a pointer to GH repo with a > >>>> prototype? > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Roman. > >>>>>>> > >>>>>>> > >>>>>>> On Sun, Feb 14, 2016 at 8:36 PM, Chinmay Kolhatkar < > >>>> [email protected]> > >>>>>>> wrote: > >>>>>>>> Hi Everyone, > >>>>>>>> > >>>>>>>> Here is some details regarding packaging apex component into > >>>> bigtop: > >>>>>>>> 1. Apex component will dependent on hadoop. This will be > >>>> mentioned in > >>>>>>>> bigtop.bom file. > >>>>>>>> 2. The component added to bom file will look like following: > >>>>>>>> > >>>>>>>> 'apex' { > >>>>>>>> name = 'apex' > >>>>>>>> relNotes = 'Apache Apex (incubating)' > >>>>>>>> version { base = '3.3.0-incubating'; pkg = base; release = > >>>> 1 } > >>>>>>>> tarball { source = > >>>>>>> "$name-${version.base}-source-release.tar.gz" > >>>>>>>> destination = "$name-${version.base}.tar.gz" } > >>>>>>>> url { download_path = > >>>> "/incubator/$name/v${version.base}/" > >>>>>>>> site = "${apache.APACHE_MIRROR}/${download_path}" > >>>>>>>> archive = > >>>> "${apache.APACHE_ARCHIVE}/${download_path}" } > >>>>>>>> } > >>>>>>>> > >>>>>>>> 3. After installation of package (deb+rpm), the directory > >>>> structure > >>>>>>> related > >>>>>>>> to apex will look like following: > >>>>>>>> > >>>>>>>> / > >>>>>>>> `-- usr > >>>>>>>> |-- bin > >>>>>>>> | |-- apex -> ../lib/apex/bin/apex > >>>>>>>> | `-- dtcli -> ../lib/apex/bin/apex > >>>>>>>> |-- lib > >>>>>>>> | `-- apex > >>>>>>>> | |-- bin > >>>>>>>> | | |-- apex > >>>>>>>> | | `-- dtcli -> apex > >>>>>>>> | |-- conf > >>>>>>>> | | `-- dt-env.sh > >>>>>>>> | `-- lib > >>>>>>>> | |-- apex-api-3.3.0-incubating.jar > >>>>>>>> | |-- apex-bufferserver-3.3.0-incubating.jar > >>>>>>>> | |-- apex-common-3.3.0-incubating.jar > >>>>>>>> | |-- apex-engine.jar > >>>>>>>> | `-- <other dependency jars> > >>>>>>>> `-- share > >>>>>>>> `-- doc > >>>>>>>> `-- apex > >>>>>>>> |-- changelog.Debian.gz > >>>>>>>> |-- copyright > >>>>>>>> `-- license > >>>>>>>> `-- LICENSE.gz > >>>>>>>> > >>>>>>>> > >>>>>>>> Please share your thoughts on above approach. > >>>>>>>> > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Chinmay. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Sat, Feb 13, 2016 at 8:38 AM, Thomas Weise <[email protected]> > >>>> wrote: > >>>>>>>> > >>>>>>>>> Chinmay, > >>>>>>>>> > >>>>>>>>> Can you share some details regarding the packaging? > >>>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> > >>>>>>>>> On Thu, Feb 11, 2016 at 12:35 AM, Chinmay Kolhatkar < > >>>>>>>>> [email protected] > >>>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Dear Community, > >>>>>>>>>> > >>>>>>>>>> Here is the Jira that is created for adding Apex into Bigtop. > >>>>>>>>>> https://issues.apache.org/jira/browse/BIGTOP-2313 > >>>>>>>>>> > >>>>>>>>>> I've started working on creating packages for Apex. > >>>>>>>>>> > >>>>>>>>>> Please share your thoughts on approach as well as on questions > >>>>>>> related to > >>>>>>>>>> packaging in my previous mail. > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> Chinmay. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Wed, Feb 10, 2016 at 9:55 PM, Chinmay Kolhatkar < > >>>>>>>>>> [email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hello Bigtop Community, > >>>>>>>>>>> > >>>>>>>>>>> Continuing with the packing effort, here is how we're > >>>> planning to > >>>>>>>>> package > >>>>>>>>>>> Apache Apex. > >>>>>>>>>>> > >>>>>>>>>>> Tentative package directory structure for apex would look > >>>> like > >>>>>>>>> following: > >>>>>>>>>>> > >>>>>>>>>>> /usr/lib/apex/bin/dtcli > >>>>>>>>>>> /usr/lib/apex/lib/apex-api-3.3.0-incubating.jar > >>>>>>>>>>> /usr/lib/apex/lib/apex-engine-3.3.0-incubating.jar > >>>>>>>>>>> /usr/lib/apex/lib/apex-bufferserver-3.3.0-incubating.jar > >>>>>>>>>>> /usr/lib/apex/lib/apex-common-3.3.0-incubating.jar > >>>>>>>>>>> /usr/lib/apex/lib/<other dependent jars> > >>>>>>>>>>> /usr/lib/apex/conf/dt-env.sh > >>>>>>>>>>> /usr/bin/dtcli -> /usr/lib/apex/bin/dtcli > >>>>>>>>>>> /usr/share/doc/man/man1/dtcli.1.gz > >>>>>>>>>>> /usr/share/doc/apex/license/LICENSE.txt > >>>>>>>>>>> /usr/share/doc/apex/license/<package>-LICENSE.txt > >>>>>>>>>>> /usr/share/doc/apex/CHANGELOG > >>>>>>>>>>> /usr/share/doc/apex/NOTICE > >>>>>>>>>>> > >>>>>>>>>>> Addition contents to bigtop.bom: > >>>>>>>>>>> 1. Dependency of apex will be on "hadoop" > >>>>>>>>>>> 2. Component addition would looks like this: > >>>>>>>>>>> 'apex' { > >>>>>>>>>>> name = 'apex' > >>>>>>>>>>> relNotes = 'Apache Apex (incubating)' > >>>>>>>>>>> version { base = '3.3.0-incubating'; pkg = base; > >>>> release = 1 > >>>>>>> } > >>>>>>>>>>> tarball { source = > >>>>>>>>>> "$name-${version.base}-source-release.tar.gz" > >>>>>>>>>>> destination = "$name-${version.base}.tar.gz" > >>>> } > >>>>>>>>>>> url { download_path = > >>>>>>> "/incubator/$name/v${version.base}/" > >>>>>>>>>>> site = > >>>> "${apache.APACHE_MIRROR}/${download_path}" > >>>>>>>>>>> archive = > >>>>>>> "${apache.APACHE_ARCHIVE}/${download_path}" } > >>>>>>>>>>> } > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> I have couple of questions related to packaging: > >>>>>>>>>>> 1. What is the best possible way to test the package locally? > >>>>>>>>>>> 2. Do you have any specific size limitation on size of the > >>>> package > >>>>>>> (rpm > >>>>>>>>>> OR > >>>>>>>>>>> deb)? > >>>>>>>>>>> 3. Is above proposed packaging information inline with bigtop > >>>>>>>>> standards? > >>>>>>>>>>> 4. Am I missing anything there? > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Chinmay. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On Fri, Feb 5, 2016 at 12:01 AM, Chinmay Kolhatkar < > >>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hello Bigtop community! > >>>>>>>>>>>> > >>>>>>>>>>>> My name is Chinmay Kolhatkar and I am a committer in Apache > >>>> Apex > >>>>>>>>>>>> (incubating) project. > >>>>>>>>>>>> Our website is at: http://apex.incubator.apache.org/ > >>>>>>>>>>>> > >>>>>>>>>>>> We want to explore and integrate Apache Apex into the > >>>> Bigtop as a > >>>>>>>>>>>> component. > >>>>>>>>>>>> > >>>>>>>>>>>> Please treat this mail as a continuation of a mail thread: > >>>> > http://mail-archives.apache.org/mod_mbox/bigtop-dev/201601.mbox/%3CCA%2B5xAo20CAHuYtdxw3mtH2X9whREKd7fLdz4xmuTMqkaTiiskg%40mail.gmail.com%3E > >>>>>>>>>>>> > >>>>>>>>>>>> We're currently confirming with requirements for components > >>>> and > >>>>>>> making > >>>>>>>>>>>> sure that we meet the expectations: > >>>> > https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution > >>>>>>>>>>>> > >>>>>>>>>>>> Once requirements are through, we're planning to add > >>>> component > >>>>>>> into > >>>>>>>>>>>> Apache Bigtop in following phases (as separate patches/pull > >>>>>>> requests): > >>>>>>>>>>>> 1. Packaging > >>>>>>>>>>>> - Creating of package i.e. rpm & deb files. > >>>>>>>>>>>> - documentations/READMEs, LICENSE, DISCLAMER, NOTES etc > >>>> if any > >>>>>>>>>> needed. > >>>>>>>>>>>> - Any documentation that need to be added to > >>>> distribution of > >>>>>>> our > >>>>>>>>>>>> software. > >>>>>>>>>>>> - Any license information of dependencies required to be > >>>>>>> added to > >>>>>>>>>>>> package > >>>>>>>>>>>> 2. Smoke tests > >>>>>>>>>>>> - Adding smoke test for packaging. > >>>>>>>>>>>> 3. Puppet recipes for automatic deployment and configuration > >>>>>>>>>>>> - Add puppet recipes for automatic deployment > >>>>>>>>>>>> 4. license clearance: > >>>>>>>>>>>> - Run 'gradle rat' to make sure all new stuff is > >>>> compliant > >>>>>>> with > >>>>>>>>> ASF > >>>>>>>>>>>> license requirements. > >>>>>>>>>>>> > >>>>>>>>>>>> Please share your thoughts on above approach. > >>>>>>>>>>>> > >>>>>>>>>>>> If the approach seems fine, we can create a Jira items and > >>>>>>>>> corresponding > >>>>>>>>>>>> subtasks to start working. > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> Chinmay. > >> >
