Author: challngr
Date: Thu Jun 13 21:44:26 2013
New Revision: 1492880
URL: http://svn.apache.org/r1492880
Log:
UIMA-2682 Duccbook updates.
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/install.tex
Modified:
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/install.tex
URL:
http://svn.apache.org/viewvc/uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/install.tex?rev=1492880&r1=1492879&r2=1492880&view=diff
==============================================================================
---
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/install.tex
(original)
+++
uima/sandbox/uima-ducc/trunk/uima-ducc-duccdocs/src/site/tex/duccbook/part4/install.tex
Thu Jun 13 21:44:26 2013
@@ -57,7 +57,8 @@ this, the following is required:
To build the documentation, the following is required:
\begin{itemize}
- \item Latex, including the \emph{pdflatex} and \emph{htlatex} packages.
+ \item Latex, including the \emph{pdflatex} and \emph{htlatex} packages. A
good place
+ to start if you need to install it is
\url{http://latex-project.org/ftp.html}
\end{itemize}
More detailed one-time setup instructions for source-level builds via
subversion can be found here:
@@ -66,7 +67,15 @@ More detailed one-time setup instruction
\subsection{Documentation}
After single-user installation, the DUCC documentation is found (in both PDF
and HTML format) in the directory
ducc\_runtime/docs. As well, the DUCC web server contains a link to the full
documentation on each major page.
+The API is documented only via JavaDoc, distributed in the web server's root
directory
+{\tt \duccruntime/webserver/root/doc/apidocs.}
+If building from source, Maven places the documentation in
+\begin{itemize}
+ \item {\tt trunk/uima-ducc-duccdocs/target/site} (main documentation), and
+ \item {\tt trunk/site/apidocs} (API Javadoc)
+\end{itemize}
+
\subsection{Single-user Installation and Verification}
Single-user installation sets up an initial, working configuration on a single
system. No security
@@ -80,11 +89,18 @@ working, one may proceed to upgrade to f
\begin{itemize}
\item One Intel-based or IBM Power-based system. (More systems may be
added during multi-user
installation, described below.)
+
\item 8GB of memory. 16GB or more is preferable for developing and
testing applications beyond
the non-trivial.
+
\item 1GB disk space to hold the DUCC runtime, system logs, and job logs.
More is
usually needed for larger installations.
- \end{itemize}
+\end{itemize}
+
+Please note: DUCC is intended for scaling out memory-intensive UIMA
applications over computing
+clusters consisting of multiple nodes with large (16GB-256GB or more) memory.
The minimal
+requirements are for initial test and evaluation purposes, but will not be
sufficient to run actual
+workloads.
\subsection{Single-user System Installation}
\begin{enumerate}
@@ -95,7 +111,7 @@ tar -zxf <distribution.file>
This creates a directory with the same name as
``$<$distribution.file$>$'', without the trailing ``.tgz''.
- This directory contains the full DUCC runtime in a subdirectory called
\duccruntime. (Note:
+ This directory contains the full DUCC runtime in a sub-directory
called \duccruntime. (Note:
the version may be different according the the actual version of DUCC
being installed.)
\item You may use the \duccruntime ``in place'' but it is highly
recommended that you move it
@@ -107,7 +123,7 @@ mv apache-uima-ducc-0.8.0-SNAPSHOT/ducc_
We refer to this directory, regardless of its location, as
\duccruntime. For simplicity,
this document assumes it is moved to ducc's \$HOME/\duccruntime.
- \item Change directories into the admin subdirectory of \duccruntime:
+ \item Change directories into the admin sub-directory of \duccruntime:
\begin{verbatim}
cd $HOME/ducc_runtime/admin
\end{verbatim}
@@ -134,15 +150,13 @@ The post-installation script performs th
\begin{enumerate}
\item Verifies that the correct level of Java and Python are installed and
available.
\item Creates a default nodelist, \duccruntime/resources/ducc.nodes,
containing the name of the node you are installing on.
- \item Establishes a nodepool for the DUCC Job Driver (JD) as the node you
are installing from.
- \item Defines the Òducc head nodeÓ to be to node you are installing from.
+ \item Defines the ``ducc head'' node to be to node you are installing from.
\item Sets up the default https keystore for the webserver.
- \item Installs the DUCC documentation Òducc bookÓ into the DUCC webserver
root.
- \item Builds and installs the C program, Òducc\_lingÓ, into the default
location.
+ \item Installs the DUCC documentation ``ducc book'' into the DUCC
webserver root.
+ \item Builds and installs the C program, ``ducc\_ling'', into the default
location.
\item Insures that the (supplied) ActiveMQ broker is runnable.
\end{enumerate}
-
\subsection{Initial System Verification}
Here we start the basic installation, submit a simple UIMA-AS job, verify that
it ran, and stop
@@ -204,7 +218,7 @@ ducchead.biz.org
bash-4.1$
\end{verbatim}
- Now open a browser and go to the DUCC webserverÕs url,
http://$<$hostname$>$:42133 where $<$hostname$>$ is
+ Now open a browser and go to the DUCC webserver's url,
http://$<$hostname$>$:42133 where $<$hostname$>$ is
the name of the host where DUCC is started. Navigate to the Reservations
page via the links in
the upper-left corner. You should see the DUCC JobDriver reservation in
state
WaitingForResources. In a few minutes this should change to Assigned.
(This usually takes 3-4
@@ -218,7 +232,7 @@ bash-4.1$
Open the browser in the DUCC jobs page. You should see the job progress
through a series of
transitions: Waiting For Driver, Waiting For Services, Waiting For
Resources, Initializing, and
- finally, Running. YouÕll see the number of work items submitted (15) and
the number of work
+ finally, Running. You'll see the number of work items submitted (15) and
the number of work
items completed grow from 0 to 15. Finally, the job will move into
Completing and then
Completed..
@@ -227,13 +241,13 @@ bash-4.1$
$HOME/ducc/logs/job-id
\end{verbatim}
- In this directory, you will find a log for the sample jobÕs JobDriver
(JD), JobProcess (JP), and
+ In this directory, you will find a log for the sample job's JobDriver
(JD), JobProcess (JP), and
a number of other files relating to the job.
This is a good time to explore the DUCC web pages. Notice that the job id
is a link to a set of
pages with details about the execution of the job.
- Notice also, in the upper-right corner is a link to the full DUCC
documentation, the ÒDuccBookÓ.
+ Notice also, in the upper-right corner is a link to the full DUCC
documentation, the ``DuccBook''.
Finally, stop DUCC:
\begin{enumerate}
@@ -254,7 +268,7 @@ $HOME/ducc/logs/job-id
In that directory are found logs for each of the DUCC components plus one
for each node DUCC is
installed on.
- DUCC job/user logs are written by default to the userÕs HOME directory
under
+ DUCC job/user logs are written by default to the user's HOME directory
under
\begin{verbatim}
$HOME/ducc/logs/<jobid>
\end{verbatim}
@@ -284,6 +298,15 @@ $HOME/ducc/logs/job-id
\end{itemize}
\subsection{Ducc\_ling Installation}
+ Before proceeding with this step, please note:
+ \begin{itemize}
+ \item This step is required ONLY to install multi-user capabilities.
+ \item The sequence operations consisting of {\em chown} and {\em
chmod} MUST be performed
+ in the exact order given below. If the {\em chmod} operation is
performed before
+ the {\em chown} operation, Linux will regress the permissions
granted by {\em chmod}
+ and ducc\_ling will be incorrectly installed.
+ \end{itemize}
+
ducc\_ling is a setuid-root program whose function is to execute user
tasks under the identity of
the user. This must be installed correctly; incorrect installation can
prevent jobs from running as
their submitters, and in the worse case, can introduce security problems
into the system.
@@ -300,7 +323,7 @@ $HOME/ducc/logs/job-id
\end{enumerate}
Now, as root, move ducc\_ling to a secure location and grant
authorization to run tasks under
- different usersÕ identities:
+ different users' identities:
\begin{enumerate}
\item mkdir /local/ducc
\item mkdir /local/ducc/bin
@@ -339,13 +362,13 @@ ducc.agent.launcher.ducc\_spawn\_path=/l
\item The next step (chown) sets ownership of
/local/ducc/bin/ducc\_ling to root, and
group ownership to ducc.
\item The next step (chmod) stablishes the {\em setuid} bit, which
allows user ducc to execute ducc\_ling
- with root priveleges.
- \item Finally, ducc.properties is updated to point to the new,
priveleged ducc\_ling.
+ with root privileges.
+ \item Finally, ducc.properties is updated to point to the new,
privileged ducc\_ling.
\end{enumerate}
If these steps are correctly performed, ONLY user {\em ducc} may use
the ducc\_ling program in
- a priveleged way. Ducc\_ling contains checks to prevent even user {\em
root} from using it for
- priveleged operations.
+ a privileged way. Ducc\_ling contains checks to prevent even user {\em
root} from using it for
+ privileged operations.
Ducc\_ling contains the following functions, which the
security-conscious may verify by examining
the source in \duccruntime/duccling. All sensitive operations are
performed only AFTER switching
@@ -365,7 +388,7 @@ ducc.agent.launcher.ducc\_spawn\_path=/l
done AFTER changing userids).
\end{itemize}
-\subsection{Set up the full nodelists (optional)}
+\subsection{Set up the full nodelists}
To add additional nodes to the ducc cluster, DUCC needs to know what nodes to
start its Agent
processes on. These nodes are listed in the file
\begin{verbatim}
@@ -378,7 +401,7 @@ cluster.
\subsection{Full DUCC Verification}
-This is identical to initial verification, with the one difference that the
job Ò1.jobÓ should be
+This is identical to initial verification, with the one difference that the
job ``1.job'' should be
submitted as any user other than ducc. Watch the webserver and insure that
you see the job execute
under the correct identity. Once this completes, DUCC is installed and
verified.
