mikewalch commented on a change in pull request #883: Fixes #842 Support multiple ways for running Fluo applications URL: https://github.com/apache/incubator-fluo/pull/883#discussion_r130673952
########## File path: docs/install.md ########## @@ -104,101 +101,66 @@ Running the script without any arguments prints a description of all commands. ./bin/fluo -## Configure a Fluo application - -You are now ready to configure a Fluo application. Use the command below to create the -configuration necessary for a new application. Feel free to pick a different name (other than -`myapp`) for your application: - - fluo new myapp - -This command will create a directory for your application at `apps/myapp` of your Fluo install which -will contain a `conf` and `lib`. - -The `apps/myapp/conf` directory contains a copy of the `fluo.properties` from your default -configuration. This should be configured for your application: - - vim apps/myapp/fluo.properties - -When configuring the observer section in fluo.properties, you can configure your instance for the -[phrasecount] application if you have not created your own application. See the [phrasecount] -example for instructions. You can also choose not to configure any observers but your workers will -be idle when started. - -The `apps/myapp/lib` directory should contain any observer jars for your application. If you -configured [fluo.properties] for observers, copy any jars containing these observer classes to this -directory. - -## Initialize your application - -After your application has been configured, use the command below to initialize it: +## Initialize Fluo application - fluo init myapp +1. Create a copy of [fluo-app.properties] for your Fluo application. -This only needs to be called once and stores configuration in Zookeeper. + cp conf/fluo-app.properties /path/to/myapp/fluo-app.properties -## Start your application +2. Edit your copy of [fluo-app.properties] and make sure to set the following: -A Fluo application consists of one oracle process and multiple worker processes. Before starting -your application, you can configure the number of worker process in your [fluo.properties] file. + * Fluo application name + * Class name of your ObserverProvider + * Paths to your Fluo observer jars + * Accumulo configuration + * HDFS configuration -When you are ready to start your Fluo application on your YARN cluster, run the command below: + When configuring the observer section of fluo-app.properties, you can configure your instance for the + [phrasecount] application if you have not created your own application. See the [phrasecount] + example for instructions. You can also choose not to configure any observers but your workers will + be idle when started. - fluo start myapp +3. Run the command below to initialize your Fluo application: -The start command above will work for a single-node or a large cluster. By using YARN, you do not -need to deploy the Fluo binaries to every node on your cluster or start processes on every node. + fluo init /path/to/myapp/fluo-app.properties -You can use the following command to check the status of your instance: + A Fluo application only needs to be initialized once. After initialization, the Fluo application + name (set by `fluo.application.name` in your [fluo-app.properties]) is used to start/stop the application + and scan the Fluo table. - fluo status myapp +4. Run `fluo list` which connects to Fluo and lists applications to verify initialization. -For more detailed information on the YARN containers running Fluo: +## Start your Fluo application - fluo info myapp +Follow the instructions below to start a Fluo application which contains an oracle and multiple workers. -You can also use `yarn application -list` to check the status of your Fluo instance in YARN. +1. Configure [fluo-env.sh] and [fluo-conn.properties] if you have not already. -## View Fluo application logs +2. Run Fluo application processes using the `fluo oracle` and `fluo worker` commands. Fluo applications + are typically run with one oracle process and multiple worker processes. The commands below will start + a Fluo oracle and two workers on your local machine: -Fluo application logs are viewable within YARN using the methods below: + fluo oracle myapp + fluo worker myapp + fluo worker myapp -1. View logs using the web interface of the the YARN resource manager -(`http://<resource manager>:8088/cluster`). First, click on the application ID (i.e `application_*`) -of your Fluo application and then click on the latest attempt ID (i.e `appattempt_*`). You should -see a list of containers. There should be a container for the application master (typically -container 1), a Fluo oracle (typically container 2), and Fluo workers (containers 3+). You can view -the log files produced by a container by clicking on its 'logs' link. Logs from Fluo observers will -be in the `worker_*.log` file for each of your worker containers. + The commands will retrieve your application configuration and observer jars (using your + application name) before starting the oracle or worker process. The commands are designed + to be run on a cluster to distribute the oracle and workers across multiple nodes. -2. View logs on disk in the directory specified by the YARN property `yarn.nodemanager.log-dirs` in -your YARN configuration `yarn-site.xml` (see [yarn-default.xml] for more info about this property). -If you are running Fluo on single machine, this method works well. If you are running Fluo on a -cluster, your containers and their logs will be spread across the cluster. +The oracle & worker logs can be found in the `logs/` directory of your Fluo installation. Review comment: fixed ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services