OOZIE-2660 Create documentation for DB Dump/Load functionality (gezapeti via rkanter)
Project: http://git-wip-us.apache.org/repos/asf/oozie/repo Commit: http://git-wip-us.apache.org/repos/asf/oozie/commit/34c469dd Tree: http://git-wip-us.apache.org/repos/asf/oozie/tree/34c469dd Diff: http://git-wip-us.apache.org/repos/asf/oozie/diff/34c469dd Branch: refs/heads/oya Commit: 34c469dd2f7023c92bc0f5605274059c30beea0c Parents: bb52d65 Author: Robert Kanter <rkan...@cloudera.com> Authored: Wed Sep 14 15:46:44 2016 -0700 Committer: Robert Kanter <rkan...@cloudera.com> Committed: Wed Sep 14 15:47:18 2016 -0700 ---------------------------------------------------------------------- distro/src/main/bin/oozie-setup.sh | 13 ++--- docs/src/site/twiki/AG_Install.twiki | 92 +++++++++++++++++++++++-------- release-log.txt | 1 + 3 files changed, 77 insertions(+), 29 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/oozie/blob/34c469dd/distro/src/main/bin/oozie-setup.sh ---------------------------------------------------------------------- diff --git a/distro/src/main/bin/oozie-setup.sh b/distro/src/main/bin/oozie-setup.sh index d3a46a9..79b049b 100644 --- a/distro/src/main/bin/oozie-setup.sh +++ b/distro/src/main/bin/oozie-setup.sh @@ -27,21 +27,22 @@ function printUsage() { echo " FS_URI is the fs.default.name" echo " for hdfs uri; SHARED_LIBRARY, path to the" echo " Oozie sharelib to install, it can be a tarball" - echo " or an expanded version of it. If ommited," + echo " or an expanded version of it. If omitted," echo " the Oozie sharelib tarball from the Oozie" echo " installation directory will be used." echo " CONCURRENCY is a number of threads to be used" echo " for copy operations." echo " By default 1 thread will be used)" - echo " (action failes if sharelib is already installed" + echo " (action fails if sharelib is already installed" echo " in HDFS)" - echo " sharelib upgrade -fs FS_URI [-locallib SHARED_LIBRARY] (upgrade existing sharelib, fails if there" + echo " sharelib upgrade -fs FS_URI [-locallib SHARED_LIBRARY] ([deprecated][use create command to create new version]" + echo " upgrade existing sharelib, fails if there" echo " is no existing sharelib installed in HDFS)" echo " db create|upgrade|postupgrade -run [-sqlfile <FILE>] (create, upgrade or postupgrade oozie db with an" echo " optional sql File)" - echo " export file exports the oozie database to the specified" + echo " export <file> exports the oozie database to the specified" echo " file in zip format" - echo " import file imports the oozie database from JSON file" + echo " import <file> imports the oozie database from the zip file" echo " created by export" echo " (without options prints this usage information)" echo @@ -166,10 +167,8 @@ do shift ${JAVA_BIN} ${OOZIE_OPTS} -cp ${OOZIECPPATH} org.apache.oozie.tools.OozieDBCLI "${@}" elif [ "$1" = "export" ]; then - shift ${JAVA_BIN} ${OOZIE_OPTS} -cp ${OOZIECPPATH} org.apache.oozie.tools.OozieDBExportCLI "${@}" elif [ "$1" = "import" ]; then - shift ${JAVA_BIN} ${OOZIE_OPTS} -cp ${OOZIECPPATH} org.apache.oozie.tools.OozieDBImportCLI "${@}" fi exit $? http://git-wip-us.apache.org/repos/asf/oozie/blob/34c469dd/docs/src/site/twiki/AG_Install.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/AG_Install.twiki b/docs/src/site/twiki/AG_Install.twiki index a03512c..0f26306 100644 --- a/docs/src/site/twiki/AG_Install.twiki +++ b/docs/src/site/twiki/AG_Install.twiki @@ -65,28 +65,32 @@ The =oozie-setup.sh= script prepares the embedded Tomcat server to run Oozie. The =oozie-setup.sh= script options are: <verbatim> -Usage : oozie-setup.sh <OPTIONS>" - prepare-war [-d directory] [-secure] (-d identifies an alternative directory for processing jars" - -secure will configure the war file to use HTTPS (SSL))" - sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] [-concurrency CONCURRENCY]" - (create sharelib for oozie," - FS_URI is the fs.default.name" - for hdfs uri; SHARED_LIBRARY, path to the" - Oozie sharelib to install, it can be a tarball" - or an expanded version of it. If omitted," - the Oozie sharelib tarball from the Oozie" - installation directory will be used." - CONCURRENCY is a number of threads to be used" - for copy operations." - By default 1 thread will be used)" - (action fails if sharelib is already installed" - in HDFS)" - sharelib upgrade -fs FS_URI [-locallib SHARED_LIBRARY] ([deprecated][use create command to create new version] - upgrade existing sharelib, fails if there" - is no existing sharelib installed in HDFS)" - db create|upgrade|postupgrade -run [-sqlfile <FILE>] (create, upgrade or postupgrade oozie db with an" - optional sql file)" - (without options prints usage information)" +Usage : oozie-setup.sh <Command and OPTIONS> + prepare-war [-d directory] [-secure] (-d identifies an alternative directory for processing jars + -secure will configure the war file to use HTTPS (SSL)) + sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] [-concurrency CONCURRENCY] + (create sharelib for oozie, + FS_URI is the fs.default.name + for hdfs uri; SHARED_LIBRARY, path to the + Oozie sharelib to install, it can be a tarball + or an expanded version of it. If omitted, + the Oozie sharelib tarball from the Oozie + installation directory will be used. + CONCURRENCY is a number of threads to be used + for copy operations. + By default 1 thread will be used) + (action fails if sharelib is already installed + in HDFS) + sharelib upgrade -fs FS_URI [-locallib SHARED_LIBRARY] ([deprecated][use create command to create new version] + upgrade existing sharelib, fails if there + is no existing sharelib installed in HDFS) + db create|upgrade|postupgrade -run [-sqlfile <FILE>] (create, upgrade or postupgrade oozie db with an + optional sql File) + export <file> exports the oozie database to the specified + file in zip format + import <file> imports the oozie database from the zip file + created by export + (without options prints this usage information) </verbatim> If a directory =libext/= is present in Oozie installation directory, the =oozie-setup.sh= script @@ -215,6 +219,50 @@ following configuration properties in the oozie-site.xml: oozie.service.JPAService.pool.max.active.conn=10 </verbatim> +---++ Database Migration + +Oozie provides an easy way to switch between databases without losing any data. Oozie servers should be stopped during the +database migraition process. +The export of the database can be done using the following command: +<verbatim> +$ bin/oozie-setup.sh export /tmp/oozie_db.zip +1 rows exported from OOZIE_SYS +50 rows exported from WF_JOBS +340 rows exported from WF_ACTIONS +10 rows exported from COORD_JOBS +70 rows exported from COORD_ACTIONS +0 rows exported from BUNDLE_JOBS +0 rows exported from BUNDLE_ACTIONS +0 rows exported from SLA_REGISTRATION +0 rows exported from SLA_SUMMARY +</verbatim> + +The database configuration is read from =oozie-site.xml=. After updating the configuration to point to the new database, +the tables have to be created with ooziedb.sh in the [[AG_Install#Database_Configuration][Database configuration]] +section above. +Once the tables are created, they can be filled with data using the following command: + +<verbatim> +$ bin/oozie-setup.sh import /tmp/oozie_db.zip +Loading to Oozie database version 3 +50 rows imported to WF_JOBS +340 rows imported to WF_ACTIONS +10 rows imported to COORD_JOBS +70 rows imported to COORD_ACTIONS +0 rows imported to BUNDLE_JOBS +0 rows imported to BUNDLE_ACTIONS +0 rows imported to SLA_REGISTRATION +0 rows imported to SLA_SUMMARY +</verbatim> + +NOTE: The database version of the zip must match the version of the Oozie database it's imported to. + +After starting the Oozie server, the history and the currently running workflows should be available. + +*IMPORTANT:* The tool was primarily developed to make the migration from embedded databases (e.g. Derby) to standalone databases + (e.g. MySQL, Posgresql, Oracle, MS SQL Server), though it will work between any supported databases. +It is *not* optimized to handle databases over 1 Gb. If the database size is larger, it should be purged before migration. + ---++ Oozie Configuration By default, Oozie configuration is read from Oozie's =conf/= directory http://git-wip-us.apache.org/repos/asf/oozie/blob/34c469dd/release-log.txt ---------------------------------------------------------------------- diff --git a/release-log.txt b/release-log.txt index 82cee6d..8833d35 100644 --- a/release-log.txt +++ b/release-log.txt @@ -1,5 +1,6 @@ -- Oozie 4.3.0 release (trunk - unreleased) +OOZIE-2660 Create documentation for DB Dump/Load functionality (gezapeti via rkanter) OOZIE-2659 TestPauseTransitService is flaky (gezapeti via rkanter) OOZIE-2488 Upgrade to latest OpenJPA version (abhishekbafna via jaydeepvishwakarma) OOZIE-2648 Child jobs shouldn't send callbacks to Oozie (abhishekbafna via rkanter)
