Hi Roman,
Thanks for the feedback.
Here are the answers to the question in the same order:
1. Apex is a YARN-native platform. Hence apex needs YARN to be there.
2. conf/ directory recently got removed. This is not needed anymore in the
package.
3. Here is the link to GH repo for apex platform:
https://github.com/apache/incubator-apex-core
The source as of now does not output any package. do-component build
would compile the apex platform and generate a rpm/deb package using files
present in source code. Hence there is no prototype of the package.
Is that the prototype you're looking for?
Thanks,
Chinmay.
On Mon, Feb 15, 2016 at 11:11 AM, Roman Shaposhnik <[email protected]>
wrote:
> Looks good to me. Three questions:
> 1. Does apex provide standalone daemon or does only run on YARN ?
> 2. What's the intended use for content under /usr/lib/apex/conf ?
> 3. Any chance you can provide a pointer to GH repo with a prototype?
>
> Thanks,
> Roman.
>
>
> On Sun, Feb 14, 2016 at 8:36 PM, Chinmay Kolhatkar <[email protected]>
> wrote:
> > Hi Everyone,
> >
> > Here is some details regarding packaging apex component into bigtop:
> > 1. Apex component will dependent on hadoop. This will be mentioned in
> > bigtop.bom file.
> > 2. The component added to bom file will look like following:
> >
> > 'apex' {
> > name = 'apex'
> > relNotes = 'Apache Apex (incubating)'
> > version { base = '3.3.0-incubating'; pkg = base; release = 1 }
> > tarball { source =
> "$name-${version.base}-source-release.tar.gz"
> > destination = "$name-${version.base}.tar.gz" }
> > url { download_path = "/incubator/$name/v${version.base}/"
> > site = "${apache.APACHE_MIRROR}/${download_path}"
> > archive = "${apache.APACHE_ARCHIVE}/${download_path}" }
> > }
> >
> > 3. After installation of package (deb+rpm), the directory structure
> related
> > to apex will look like following:
> >
> > /
> > `-- usr
> > |-- bin
> > | |-- apex -> ../lib/apex/bin/apex
> > | `-- dtcli -> ../lib/apex/bin/apex
> > |-- lib
> > | `-- apex
> > | |-- bin
> > | | |-- apex
> > | | `-- dtcli -> apex
> > | |-- conf
> > | | `-- dt-env.sh
> > | `-- lib
> > | |-- apex-api-3.3.0-incubating.jar
> > | |-- apex-bufferserver-3.3.0-incubating.jar
> > | |-- apex-common-3.3.0-incubating.jar
> > | |-- apex-engine.jar
> > | `-- <other dependency jars>
> > `-- share
> > `-- doc
> > `-- apex
> > |-- changelog.Debian.gz
> > |-- copyright
> > `-- license
> > `-- LICENSE.gz
> >
> >
> > Please share your thoughts on above approach.
> >
> >
> > Thanks,
> > Chinmay.
> >
> >
> >
> >
> > On Sat, Feb 13, 2016 at 8:38 AM, Thomas Weise <[email protected]> wrote:
> >
> >> Chinmay,
> >>
> >> Can you share some details regarding the packaging?
> >>
> >> Thanks
> >>
> >> On Thu, Feb 11, 2016 at 12:35 AM, Chinmay Kolhatkar <
> >> [email protected]
> >> > wrote:
> >>
> >> > Dear Community,
> >> >
> >> > Here is the Jira that is created for adding Apex into Bigtop.
> >> > https://issues.apache.org/jira/browse/BIGTOP-2313
> >> >
> >> > I've started working on creating packages for Apex.
> >> >
> >> > Please share your thoughts on approach as well as on questions
> related to
> >> > packaging in my previous mail.
> >> >
> >> > Thanks,
> >> > Chinmay.
> >> >
> >> >
> >> >
> >> > On Wed, Feb 10, 2016 at 9:55 PM, Chinmay Kolhatkar <
> >> > [email protected]>
> >> > wrote:
> >> >
> >> > > Hello Bigtop Community,
> >> > >
> >> > > Continuing with the packing effort, here is how we're planning to
> >> package
> >> > > Apache Apex.
> >> > >
> >> > > Tentative package directory structure for apex would look like
> >> following:
> >> > >
> >> > > /usr/lib/apex/bin/dtcli
> >> > > /usr/lib/apex/lib/apex-api-3.3.0-incubating.jar
> >> > > /usr/lib/apex/lib/apex-engine-3.3.0-incubating.jar
> >> > > /usr/lib/apex/lib/apex-bufferserver-3.3.0-incubating.jar
> >> > > /usr/lib/apex/lib/apex-common-3.3.0-incubating.jar
> >> > > /usr/lib/apex/lib/<other dependent jars>
> >> > > /usr/lib/apex/conf/dt-env.sh
> >> > > /usr/bin/dtcli -> /usr/lib/apex/bin/dtcli
> >> > > /usr/share/doc/man/man1/dtcli.1.gz
> >> > > /usr/share/doc/apex/license/LICENSE.txt
> >> > > /usr/share/doc/apex/license/<package>-LICENSE.txt
> >> > > /usr/share/doc/apex/CHANGELOG
> >> > > /usr/share/doc/apex/NOTICE
> >> > >
> >> > > Addition contents to bigtop.bom:
> >> > > 1. Dependency of apex will be on "hadoop"
> >> > > 2. Component addition would looks like this:
> >> > > 'apex' {
> >> > > name = 'apex'
> >> > > relNotes = 'Apache Apex (incubating)'
> >> > > version { base = '3.3.0-incubating'; pkg = base; release = 1 }
> >> > > tarball { source =
> >> > "$name-${version.base}-source-release.tar.gz"
> >> > > destination = "$name-${version.base}.tar.gz" }
> >> > > url { download_path = "/incubator/$name/v${version.base}/"
> >> > > site = "${apache.APACHE_MIRROR}/${download_path}"
> >> > > archive =
> "${apache.APACHE_ARCHIVE}/${download_path}" }
> >> > > }
> >> > >
> >> > >
> >> > > I have couple of questions related to packaging:
> >> > > 1. What is the best possible way to test the package locally?
> >> > > 2. Do you have any specific size limitation on size of the package
> (rpm
> >> > OR
> >> > > deb)?
> >> > > 3. Is above proposed packaging information inline with bigtop
> >> standards?
> >> > > 4. Am I missing anything there?
> >> > >
> >> > > Thanks,
> >> > > Chinmay.
> >> > >
> >> > >
> >> > >
> >> > > On Fri, Feb 5, 2016 at 12:01 AM, Chinmay Kolhatkar <
> >> > > [email protected]> wrote:
> >> > >
> >> > >> Hello Bigtop community!
> >> > >>
> >> > >> My name is Chinmay Kolhatkar and I am a committer in Apache Apex
> >> > >> (incubating) project.
> >> > >> Our website is at: http://apex.incubator.apache.org/
> >> > >>
> >> > >> We want to explore and integrate Apache Apex into the Bigtop as a
> >> > >> component.
> >> > >>
> >> > >> Please treat this mail as a continuation of a mail thread:
> >> > >>
> >> > >>
> >> >
> >>
> http://mail-archives.apache.org/mod_mbox/bigtop-dev/201601.mbox/%3CCA%2B5xAo20CAHuYtdxw3mtH2X9whREKd7fLdz4xmuTMqkaTiiskg%40mail.gmail.com%3E
> >> > >>
> >> > >> We're currently confirming with requirements for components and
> making
> >> > >> sure that we meet the expectations:
> >> > >>
> >> > >>
> >> >
> >>
> https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution
> >> > >>
> >> > >> Once requirements are through, we're planning to add component into
> >> > >> Apache Bigtop in following phases (as separate patches/pull
> requests):
> >> > >> 1. Packaging
> >> > >> - Creating of package i.e. rpm & deb files.
> >> > >> - documentations/READMEs, LICENSE, DISCLAMER, NOTES etc if any
> >> > needed.
> >> > >> - Any documentation that need to be added to distribution of
> our
> >> > >> software.
> >> > >> - Any license information of dependencies required to be added
> to
> >> > >> package
> >> > >> 2. Smoke tests
> >> > >> - Adding smoke test for packaging.
> >> > >> 3. Puppet recipes for automatic deployment and configuration
> >> > >> - Add puppet recipes for automatic deployment
> >> > >> 4. license clearance:
> >> > >> - Run 'gradle rat' to make sure all new stuff is compliant with
> >> ASF
> >> > >> license requirements.
> >> > >>
> >> > >> Please share your thoughts on above approach.
> >> > >>
> >> > >> If the approach seems fine, we can create a Jira items and
> >> corresponding
> >> > >> subtasks to start working.
> >> > >>
> >> > >> Thanks,
> >> > >> Chinmay.
> >> > >>
> >> > >>
> >> > >
> >> >
> >>
>
