Author: stevel
Date: Thu Sep 4 20:37:02 2014
New Revision: 1622558
URL: http://svn.apache.org/r1622558
Log:
SLIDER-86 update docs on windows
Modified:
incubator/slider/site/trunk/content/developing/building.md
incubator/slider/site/trunk/content/developing/windows.md
Modified: incubator/slider/site/trunk/content/developing/building.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/developing/building.md?rev=1622558&r1=1622557&r2=1622558&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/developing/building.md (original)
+++ incubator/slider/site/trunk/content/developing/building.md Thu Sep 4
20:37:02 2014
@@ -28,9 +28,9 @@ The network on the development system mu
of the local host working. Tests will fail without this. For maven builds to
work, remote network access is often a pre-requisite.
-### Java 7
+### Java
-Slider is built on Java 7 -please have a JDK for Java 7 or 8 set up
+Slider is built on Java 6 or later
### Python
@@ -60,7 +60,10 @@ This is absolutely critical to prevent J
mvn clean install
-Slider code base includes unit tests and functional tests. By default
functional tests are not run as these tests are developed to be run against
live hadoop clusters and require some manual setup. You can run them based as
described at [functional test](/developing/functional_tests.html).
+Slider code base includes unit tests and functional tests.
+By default functional tests are not run as these tests are developed to be run
+against live hadoop clusters and require some manual setup.
+You can run them based as described at [functional
test](/developing/functional_tests.html).
### Create Slider Package
@@ -68,7 +71,9 @@ Slider code base includes unit tests and
mvn clean site:site site:stage package -DskipTests
-***At this point you are ready to use the Slider toolset. The build
instructions below are optional and are proven to be useful when debugging deep
into the hadoop code base.***
+***At this point you are ready to use the Slider toolset.
+The build instructions below are optional and are proven to
+be useful when debugging deep into the hadoop code base.***
----------------
@@ -85,25 +90,24 @@ to have a local version of Hadoop -so th
Hadoop as well in Slider.
-To build and install locally, check out apache svn/github, branch
`release-2.4.0`,
+To build and install locally, check out apache svn/github, branch `branch-2`,
and create a branch off that tag
git clone git://git.apache.org/hadoop-common.git
cd hadoop-common
- git remote rename origin apache
- git fetch --tags apache
- git checkout release-2.4.0 --
- git checkout -b release-2.4.0
+ git fetch --tags origin
+ git checkout -b origin/branch-2
+
+ git checkout -b branch-2
+ export HADOOP_VERSION=2.6.0-SNAPSHOT
-For the scripts below, set the `HADOOP_VERSION` variable to the version
+(to build against a release, check out that specific release and create
+a branch off it:)
+
+ git checkout release-2.6.0 --
+ git checkout -b release-2.6.0
- export HADOOP_VERSION=2.4.0
-
-or, for building against a pre-release version of Hadoop 2.4
-
- git checkout branch-2
- export HADOOP_VERSION=2.4.0-SNAPSHOT
To build and install it locally, skipping the tests:
@@ -117,13 +121,14 @@ To make a tarball for use in test runs:
# on linux
mvn clean package -Pdist -Pnative -Dtar -DskipTests
-Dmaven.javadoc.skip=true
-Then expand this
+Then clean expand this
pushd hadoop-dist/target/
- gunzip hadoop-$HADOOP_VERSION.tar.gz
- tar -xvf hadoop-$HADOOP_VERSION.tar
+ tar -xvzf hadoop-$HADOOP_VERSION.tar.gz
popd
+
+
This creates an expanded version of Hadoop. You can now actually run Hadoop
from this directory. Do note that unless you have the native code built for
your target platform, Hadoop will be slower.
@@ -393,7 +398,8 @@ either stream for logging
Example:
-
target/TestLiveRegionService/TestLiveRegionService-logDir-nm-1_0/application_1376095770244_0001/container_1376095770244_0001_01_000001/out.txt
+ target/TestLiveRegionService/TestLiveRegionService-logDir-nm-1_0/
+
application_1376095770244_0001/container_1376095770244_0001_01_000001/out.txt
1. The actual test log from JUnit itself goes to the console and into
`target/surefire/`; this shows the events happening in the YARN services as
well
Modified: incubator/slider/site/trunk/content/developing/windows.md
URL:
http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/developing/windows.md?rev=1622558&r1=1622557&r2=1622558&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/developing/windows.md (original)
+++ incubator/slider/site/trunk/content/developing/windows.md Thu Sep 4
20:37:02 2014
@@ -31,20 +31,39 @@ This documents what you need to build an
## Virtual Box
-1. Ask for lots of RAM
-2. 2D acceleration of video
+1. Ask for lots of RAM, CPU. You can vary CPU and limits as you go, and
+increase them for test runs/full screen work.
+1. 2D acceleration of video
1. Critical: Enable PAE under "processor". this avoids random hangs.
+
+### OS features
+
+In the server management app, go to "manage features", where you can add
+
+* Telnet server: lets you telnet in.
+* Maybe: desktop experience
+
+### OS settings
+
+Firewall setup is a mystery. You need to convince the OS that your Ether
network
+is "private", then edit the firewall to open it up.
+
## Dev tools
1. Bitvise SSH server; add your private key to the account you create.
+ (even with this installed, you will still be mostly working from the windows
+ command prompt on the machine, as it has the env variables set up)
1. Git
1. Java 7
-1. Everything Hadoop's `BUILDING.TXT` needs. This includes the gnu tools,
protoc,
+1. Python 2,7
+1. OpenSSL
+1. Everything Hadoop's `BUILDING.TXT` needs. This includes the cygwin tools,
protoc,
python and more. If you cannot build Hadoop from the `Windows SDK Command
Prompt`
then don't proceed.
1. PFE32 is still one of the best plain text editors for development around,
even just for viewing logs & test results.
+1. ...JEdit is fairly nimble too, though not as lightweight.
1. Chrome can be used a browser instead of IE if you want âbut you should
still
view all web pages in IE to verify they look OK.
@@ -60,4 +79,133 @@ view all web pages in IE to verify they
1. then the user env vars ... scroll down to find path.
1. copy the `Path` value, edit in PFE or similar, paste back in.
1. Open a new command window.
-
\ No newline at end of file
+
+
+## Windows
+
+1. Follow the Hadoop `BUILDING.TXT` instructions.
+1. install cygwin64 with it's `sh` command.
+1. Get the cygwin64 bin dir on the PATH (remember, ";" as separators)
+1. find where the `CL` C/C++ compiler is and make sure it is on the PATH
+1. get the (native) [CMake 2.8+](http://www.cmake.org/) command on the path,
+not the cygwin one.
+
+
+Things to install in cygwin
+* sh
+* ls
+* rm
+* cp
+* gzip
+* tar
+* awk
+* grep
+* xargs
+
+
+
+## Avoid
+
+* having cygwin `cmake` on your path; it must be the native cmake
+
+### env variables
+
+ CYGWIN=nodosfilewarning
+ PATH = cmake bin, cygwin bin, python bin, hadoop_home bin (see below)
+
+example:
+
+ C:\bin\cygwin64\bin;C:\bin\Python27\;C:\Windows\system32;C:\Windows;
+ C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;
+ c:\java\jdk7\bin;C:\Program Files\Microsoft Windows Performance Toolkit\;
+ C:\apps\maven\bin;C:\apps\Git\cmd;c:\bin;
+ C:\Program Files (x86)\MSBuild\12.0\Bin;
+ C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin;
+ C:\bin\cmake-2.8\bin;C:\bin\hadoop\bin;C:\bin\OpenSSL\bin
+
+
+### Commands to test CLI setup
+
+ cmake --version
+ openssl version
+ python --version
+ git --version
+ tar --version
+
+ java -version
+ mvn --version
+
+ msbuild /version
+ cl /help
+
+and for hadoop, once installed
+
+winutils sysinfo
+hadoop version
+hadoop checknative
+
+
+
+### Slider's test for windows support
+
+You can test for windows support with the slider-core test
+
+ mvn test -Dtest=TestWindowsSupport
+
+This looks for the arguments and other valid settings
+
+## Building Hadoop
+
+1. Find the "Windows SDK 7.1 command prompt" using the search field in the
+windows 8 screen
+1. Run this, and "pin" it to the task bar. This will be the primary console
+for building hadoop.
+1. Right mouse button on the icon in the top-left corner to bring up the
"defaults"
+dialog.
+1. Choose whatever colors and fonts you want
+1. In layout, select a wider window size than 80 (if you want)
+1. in "screen buffer", set the no. of lines to 999.
+1. close the window and re-open it to get the new settings
+
+From a Windows SDK command prompt:
+
+ set HADOOP_VERSION=2.6.0-SNAPSHOT
+
+or
+
+ set HADOOP_VERSION=3.0.0-SNAPSHOT
+
+
+in the root directory of the hadoop repository
+
+
+ mvn clean
+ mvn install package -Pdist -Pnative-win -Dtar -DskipTests
-Dmaven.javadoc.skip=true
+
+(use two commands as sometimes clean can fail with some recursive
directory-in-use
+exception)
+
+
+ pushd hadoop-dist\target
+ tar -xvzf hadoop-%HADOOP_VERSION%.tar.gz
+ rm -rf c:\bin\hadoop
+ cp -r hadoop-%HADOOP_VERSION% c:\bin\hadoop
+ popd
+
+ Hadoop is now installed into `c:\bin\%HADOOP_VERSION%`
+
+ Set up the env variables by hand
+
+ set HADOOP_HOME=C:\bin\hadoop
+ set PATH=C:\bin\hadoop\bin;%PATH%
+
+Verify the version is as expected
+
+ hadoop version
+
+Verify that the binaries are there
+
+ hadoop checknative -a
+
+ This should report the hadoop binaries, even if it warns about missing
+ compression libraries
\ No newline at end of file
