incubator-trafodion git commit: Merging [TRAFODION-2642] dcs install with enable HA
Repository: incubator-trafodion
Updated Branches:
refs/heads/master 7f60e7c50 -> 8e8896042
Merging [TRAFODION-2642] dcs install with enable HA
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit:
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/8e889604
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/8e889604
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/8e889604
Branch: refs/heads/master
Commit: 8e8896042b4a1a235421cdf8382f5a069e9269de
Parents: 7f60e7c
Author: Steve Varnau
Authored: Tue Jun 27 16:41:27 2017 +
Committer: Steve Varnau
Committed: Tue Jun 27 16:41:27 2017 +
--
core/sqf/sysinstall/home/trafodion/.bashrc | 2 +-
core/trafodion.spec | 4 ++-
.../src/asciidoc/_chapters/ambari_install.adoc | 29 ++--
.../src/asciidoc/_chapters/requirements.adoc| 18
install/ambari-installer/mpack.json | 3 ++
.../TRAFODION/2.1/configuration/dcs-env.xml | 1 +
.../TRAFODION/2.1/package/scripts/params.py | 1 +
.../2.1/package/scripts/trafodionmaster.py | 2 +-
.../python-installer/scripts/traf_sqconfig.py | 8 ++
9 files changed, 45 insertions(+), 23 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8e889604/core/sqf/sysinstall/home/trafodion/.bashrc
--
diff --git a/core/sqf/sysinstall/home/trafodion/.bashrc
b/core/sqf/sysinstall/home/trafodion/.bashrc
index 076eb29..ff5da6b 100755
--- a/core/sqf/sysinstall/home/trafodion/.bashrc
+++ b/core/sqf/sysinstall/home/trafodion/.bashrc
@@ -50,7 +50,7 @@ if [ -f $TRAF_HOME/sqenv.sh ]; then
source ./sqenv.sh
popd >/dev/null
export MANPATH=$MANPATH:$MPI_ROOT/share/man
-setup_my_nodes
+setup_my_nodes 2>/dev/null
fi
#---
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8e889604/core/trafodion.spec
--
diff --git a/core/trafodion.spec b/core/trafodion.spec
index f135cc7..4ecc873 100644
--- a/core/trafodion.spec
+++ b/core/trafodion.spec
@@ -96,7 +96,9 @@ then
fi
if ! getent passwd trafodion > /dev/null
then
- /usr/sbin/useradd --shell /bin/bash -m trafodion -g trafodion --home
/home/trafodion > /dev/null 2>&1
+ /usr/sbin/useradd --shell /bin/bash -m trafodion -g trafodion -G hbase,hive
--home /home/trafodion > /dev/null 2>&1
+else
+ /usr/sbin/usermod -a -G hbase,hive trafodion > /dev/null 2>&1
fi
chmod go+rx /home/trafodion
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8e889604/docs/provisioning_guide/src/asciidoc/_chapters/ambari_install.adoc
--
diff --git a/docs/provisioning_guide/src/asciidoc/_chapters/ambari_install.adoc
b/docs/provisioning_guide/src/asciidoc/_chapters/ambari_install.adoc
index f8b9296..fee825e 100644
--- a/docs/provisioning_guide/src/asciidoc/_chapters/ambari_install.adoc
+++ b/docs/provisioning_guide/src/asciidoc/_chapters/ambari_install.adoc
@@ -102,11 +102,12 @@ Using Ambari, select the cluster and then from the
Actions menu choose "Add Serv
image:{images}/amb3.PNG[width=300,height=300,alt="Add Service"]
-You may get a configuration warning regarding a hbase-site setting. In this
case, follow the warning message
-instructions to go to the HBase advanced configuration, hbase-site section and
use the button to revert to
+WARNING: You may get a configuration warning regarding a hbase-site setting.
In this case, follow the warning message
+instructions to go back to the HBase advanced configuration, hbase-site
section and use the button to revert to
the recommended value for "hbase.coprocessor.region.classes".
Several services will need to be restarted before the {project-name} service
can be started.
+Choose cluster "Actions" menu option "Restart All Required".
== Initialize Meta-Data
@@ -115,3 +116,27 @@ and then from the Service Actions menu choose "Initialize"
and confirm the actio
image:{images}/amb5.PNG[width=300,height=250,alt="Initialize"]
+== Upgrade from Prior Version
+
+First, be sure to stop the trafodion service prior to upgrading.
+
+Upgrade the traf_ambari package and re-start ambari-server service.
+
+ $ sudo ambari-server restart
+
+ Upgrade Entire HDP Stack
+
+Ambari requires version upgrades to be an entire stack, not single components.
If you want to
+include a trafodion upgrade in entire stack version upgrade, first upgrade the
traf_ambari
+package and restart Ambari server.
+
+Once that is done, proceed with the upgrade following Ambari instructions. Be
sure to
+define the Trafodion-2.2 URL that
incubator-trafodion git commit: Merge [TRAFODION-2626] Make conf directory location configurable
Repository: incubator-trafodion
Updated Branches:
refs/heads/master 8e8896042 -> 9e31984ea
Merge [TRAFODION-2626] Make conf directory location configurable
This closes #1129
Squash merge of:
commit c679ffdd235d3f53d6f502931f5ab970d938819d
Author: Steve Varnau
Date: Mon Jun 26 21:00:48 2017 +
[TRAFODION-2626] Updated to include recent additions to conf
Updated to modify recently added references to conf dir in jira
TRAFODION-2654.
Author: Steve Varnau
Date: Mon Jun 26 20:51:24 2017 +
Merge remote branch 'origin/master' into traf2626_conf
commit b0dcf05aa2b2580700b214dfb12543ff0a1102df
Author: Steve Varnau
Date: Tue Jun 20 16:27:44 2017 +
[TRAFODION-2626] Make conf directory location configurable
For user configuration files, use TRAF_CONF variable instead of
TRAF_HOME/conf.
In cases where we want to install config files to an alternate location
(say TRAF_CONF=/etc/trafodion), we would copy all the default files from
TRAF_HOME/conf to TRAF_CONF and make modifications in the TRAF_CONF
location.
In development workspace, both locations can be one in the same.
One case where I left the original path is for the install_features file.
This file identifies version features and is not a user-configurable file.
Hence it should be read from the TRAF_HOME/conf location.
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit:
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/9e31984e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/9e31984e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/9e31984e
Branch: refs/heads/master
Commit: 9e31984ea6118b50866134ff3490cf309e5e1ea7
Parents: 8e88960
Author: Steve Varnau
Authored: Tue Jun 27 16:52:57 2017 +
Committer: Steve Varnau
Committed: Tue Jun 27 16:55:43 2017 +
--
core/dbsecurity/auth/src/authEvents.cpp | 2 +-
core/sqf/commonLogger/CommonLogger.cpp | 17 ++--
core/sqf/conf/trafodion.udr.config | 2 +-
core/sqf/sqenvcom.sh| 1 +
core/sqf/sql/scripts/genms | 3 ++
.../seatrans/tm/hbasetmlib2/TMInterface.java| 6 +--
.../java/org/trafodion/dtm/HBaseTxClient.java | 4 +-
core/sql/sqludr/SqlUdrPredefLogReader.cpp | 42 +---
.../java/org/trafodion/sql/HBaseClient.java | 4 +-
.../main/java/org/trafodion/sql/HiveClient.java | 2 +-
.../org/trafodion/sql/SequenceFileReader.java | 2 +-
.../org/trafodion/sql/TrafConfiguration.java| 2 +-
core/sql/udrserv/UdrCfgParser.cpp | 4 +-
core/sql/udrserv/udrserv.cpp| 2 +-
14 files changed, 24 insertions(+), 69 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/9e31984e/core/dbsecurity/auth/src/authEvents.cpp
--
diff --git a/core/dbsecurity/auth/src/authEvents.cpp
b/core/dbsecurity/auth/src/authEvents.cpp
index 361a148..b63d2e5 100644
--- a/core/dbsecurity/auth/src/authEvents.cpp
+++ b/core/dbsecurity/auth/src/authEvents.cpp
@@ -37,7 +37,7 @@ static std::string AUTH_COMPONENT = "DBSECURITY";
// This function inserts an AuthEvent into the current list of authEvents
//
// DB_SECUITY_EVENTID - is the event ID to insert
-//see: $TRAF_HOME/log4cxx.trafodion.auth.config/evl_sqlog_eventnum.h
+//see: $TRAF_HOME/export/include/common/evl_sqlog_eventnum.h
// eventText - text to insert
// severity - severity of the event
//
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/9e31984e/core/sqf/commonLogger/CommonLogger.cpp
--
diff --git a/core/sqf/commonLogger/CommonLogger.cpp
b/core/sqf/commonLogger/CommonLogger.cpp
index 8a049ee..b49623a 100644
--- a/core/sqf/commonLogger/CommonLogger.cpp
+++ b/core/sqf/commonLogger/CommonLogger.cpp
@@ -58,20 +58,18 @@ bool CommonLogger::initLog4cxx(const char* configFileName,
const char* fileSuffi
logFolder_ = "";
- // Form the config path $TRAF_HOME/conf
- char *installPath = getenv("TRAF_HOME");
+ // Form the config path $TRAF_CONF
+ char *installPath = getenv("TRAF_CONF");
if (installPath)
{
configPath = installPath;
// Last char may or may not be a slash.
if (configPath[configPath.length()-1] != SLASH_C)
configPath += SLASH_C; // LCOV_EXCL_LINE :nu
-
-configPath += "conf" SLASH_S;
}
else
{
-// Environment variable $TRAF_HOME not found.
+// Environment variable $TRAF_CONF not found.
// use the current directory.
configPath = "";
}
@@ -81,15 +79,6 @@ bool CommonLogger::initLog4cx
[2/3] incubator-trafodion git commit: Fix for privs2/TEST144 failure.
Fix for privs2/TEST144 failure. The updated help text shows up as a diff in this test. I should have noticed this and included the change with the changes to the library management SPJs. Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/af0235ca Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/af0235ca Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/af0235ca Branch: refs/heads/master Commit: af0235cae9e75b6f7b8850b03ae5761d79504a99 Parents: 13cb046 Author: Hans Zeller Authored: Tue Jun 27 16:37:01 2017 + Committer: Hans Zeller Committed: Tue Jun 27 16:37:01 2017 + -- core/sql/regress/privs2/EXPECTED144 | Bin 59453 -> 59453 bytes 1 file changed, 0 insertions(+), 0 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/af0235ca/core/sql/regress/privs2/EXPECTED144 -- diff --git a/core/sql/regress/privs2/EXPECTED144 b/core/sql/regress/privs2/EXPECTED144 index 70d088f..1e13d02 100644 Binary files a/core/sql/regress/privs2/EXPECTED144 and b/core/sql/regress/privs2/EXPECTED144 differ
[1/3] incubator-trafodion git commit: [TRAFODION-2637] Part 2: Rework
Repository: incubator-trafodion
Updated Branches:
refs/heads/master 9e31984ea -> 665ea9330
[TRAFODION-2637] Part 2: Rework
This is cleanup and rework for an earlier PR that somehow got messed
up. The first commit of PR 1141 got merged into Master without
indicating so in the PR.
I am submitting a new commit with the rework from the review comments
here.
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit:
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/13cb0468
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/13cb0468
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/13cb0468
Branch: refs/heads/master
Commit: 13cb04682cba886c6e7504bff6c5468836f8384a
Parents: c7dfd5a
Author: Hans Zeller
Authored: Mon Jun 26 23:57:29 2017 +
Committer: Hans Zeller
Committed: Mon Jun 26 23:57:29 2017 +
--
.../src/main/java/org/trafodion/libmgmt/SyncLibUDF.java | 7 ---
core/sql/udrserv/udrserv.cpp | 6 --
2 files changed, 4 insertions(+), 9 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/13cb0468/core/sql/lib_mgmt/src/main/java/org/trafodion/libmgmt/SyncLibUDF.java
--
diff --git
a/core/sql/lib_mgmt/src/main/java/org/trafodion/libmgmt/SyncLibUDF.java
b/core/sql/lib_mgmt/src/main/java/org/trafodion/libmgmt/SyncLibUDF.java
index 3f8daf0..523f964 100644
--- a/core/sql/lib_mgmt/src/main/java/org/trafodion/libmgmt/SyncLibUDF.java
+++ b/core/sql/lib_mgmt/src/main/java/org/trafodion/libmgmt/SyncLibUDF.java
@@ -43,10 +43,11 @@ public class SyncLibUDF extends UDR {
public void describeParamsAndColumns(UDRInvocationInfo info)
throws UDRException
{
-// this TMUDF takes no table-valued inputs, two string
+// This TMUDF takes no table-valued inputs, two string
// parameters, and generates a table with a single integer
-// value. It assumes that it was created with the following
-// DDL, without specifying parameters or return values:
+// column (one row per parallel instance). It assumes that it
+// was created with the following DDL, without specifying
+// parameters or return values:
//
// create table_mapping function SyncLibUDF()
// external name 'org.trafodion.libmgmt.SyncLibUDF'
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/13cb0468/core/sql/udrserv/udrserv.cpp
--
diff --git a/core/sql/udrserv/udrserv.cpp b/core/sql/udrserv/udrserv.cpp
index 0a3629a..29654b1 100644
--- a/core/sql/udrserv/udrserv.cpp
+++ b/core/sql/udrserv/udrserv.cpp
@@ -176,12 +176,6 @@ static Int32 processCommandsFromFile(const char *filename,
UdrGlobals &glob);
static Int32 processSingleCommandFromFile(FILE *f, UdrGlobals &glob);
// LCOV_EXCL_STOP
-// Changed the default to 512 to limit java heap size used by SQL processes.
-// Keep this define in sync with executor/JavaObjectInterface.cpp
-#define DEFAULT_JVM_MAX_HEAP_SIZE 512
-#define DEFAULT_COMPRESSED_CLASSSPACE_SIZE 128
-#define DEFAULT_MAX_METASPACE_SIZE 128
-
static NAString initErrText("");
/*
Helper function to propagate all Java-related environment settings
[3/3] incubator-trafodion git commit: Merge [TRAFODION-2637] Part 2: Rework PR-1147
Merge [TRAFODION-2637] Part 2: Rework PR-1147 Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/665ea933 Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/665ea933 Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/665ea933 Branch: refs/heads/master Commit: 665ea93300f1d63d9cad800173e2754c66444e3a Parents: 9e31984 af0235c Author: Hans Zeller Authored: Tue Jun 27 18:18:41 2017 + Committer: Hans Zeller Committed: Tue Jun 27 18:18:41 2017 + -- .../java/org/trafodion/libmgmt/SyncLibUDF.java | 7 --- core/sql/regress/privs2/EXPECTED144 | Bin 59453 -> 59453 bytes core/sql/udrserv/udrserv.cpp| 6 -- 3 files changed, 4 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/665ea933/core/sql/udrserv/udrserv.cpp --
[2/2] incubator-trafodion git commit: merge pr-1148 Add java files needed for TEST125
merge pr-1148 Add java files needed for TEST125 Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/c401633e Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/c401633e Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/c401633e Branch: refs/heads/master Commit: c401633eb9ee93100b7cf942d058902e91731aaf Parents: 665ea93 e67e90f Author: Roberta Marton Authored: Tue Jun 27 18:47:31 2017 + Committer: Roberta Marton Committed: Tue Jun 27 18:47:31 2017 + -- core/sql/regress/privs1/TEST125 | 4 +- core/sql/regress/privs1/TestHive.java | 91 ++ core/sql/regress/privs1/Utils.java| 279 + 3 files changed, 371 insertions(+), 3 deletions(-) --
[1/2] incubator-trafodion git commit: Add java files needed for TEST125
Repository: incubator-trafodion
Updated Branches:
refs/heads/master 665ea9330 -> c401633eb
Add java files needed for TEST125
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit:
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/e67e90f8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/e67e90f8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/e67e90f8
Branch: refs/heads/master
Commit: e67e90f8bd94f8022d98904d7105f2c782f57080
Parents: 012d654
Author: Roberta Marton
Authored: Tue Jun 27 15:32:39 2017 +
Committer: Roberta Marton
Committed: Tue Jun 27 15:32:39 2017 +
--
core/sql/regress/privs1/TEST125 | 4 +-
core/sql/regress/privs1/TestHive.java | 91 ++
core/sql/regress/privs1/Utils.java| 279 +
3 files changed, 371 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/e67e90f8/core/sql/regress/privs1/TEST125
--
diff --git a/core/sql/regress/privs1/TEST125 b/core/sql/regress/privs1/TEST125
index 09e0d2d..0025c53 100644
--- a/core/sql/regress/privs1/TEST125
+++ b/core/sql/regress/privs1/TEST125
@@ -160,9 +160,6 @@ create view games_by_player as
order by player_name, team_number;
-- create function to display bitmaps as a bitmap rather than longs
--- set envvar REGRRUNDIR
'/mnt/rmarton/gitws/incubator-trafodion/core/sql/regress/rundir/privs1';
--- set envvar REGRTSTDIR
'/mnt/rmarton/gitws/incubator-trafodion/core/sql/regress/privs1';
--- set envvar scriptsdir
'/mnt/rmarton/gitws/incubator-trafodion/core/sql/regress';
sh rm -f ./etest141.dll;
sh sh $$scriptsdir$$/tools/dll-compile.ksh etest141.cpp
2>&1 | tee LOG125-SECONDARY;
@@ -175,6 +172,7 @@ language c parameter style sql external name
'translateBitmap'
library t125_l1
deterministic no sql final call allow any parallelism state area size 1024 ;
+-- create procedure that accesses hive tables
sh sh $$scriptsdir$$/tools/java-compile.ksh Utils.java TestHive.java 2>
LOG125-SECONDARY | tee -a LOG125;
sh sh $$scriptsdir$$/tools/java-archive.ksh TEST125_procs.jar TestHive.class
Utils.class 2>> LOG125-SECONDARY | tee -a LOG125;
set pattern $$JARF$$ TEST125_procs.jar;
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/e67e90f8/core/sql/regress/privs1/TestHive.java
--
diff --git a/core/sql/regress/privs1/TestHive.java
b/core/sql/regress/privs1/TestHive.java
new file mode 100644
index 000..352f226
--- /dev/null
+++ b/core/sql/regress/privs1/TestHive.java
@@ -0,0 +1,91 @@
+import java.io.*;
+import java.sql.*;
+import java.util.*;
+
+// ===
+// Class: testHive
+//
+// ===
+public class TestHive
+{
+ // --
+ // Main code for credentials procedure
+ // --
+ public static void accessHive (
+String operationIn,
+String[] results)
+
+ throws SQLException
+ {
+// Initialize the Utils structure
+// Utils provides connection, user, current time, and logging information
+Utils util = null;
+
+try
+{
+ util = new Utils();
+}
+catch(Exception e)
+{
+ String theError = "ERROR: unable to create the Util object";
+ throw e;
+}
+
+util.log ("");
+util.log ("* Starting credentials request *");
+
+// Set up the connection
+Connection conn = null;
+String sessionUser = null;
+String currentUser = null;
+String currentTime = null;
+try
+{
+ conn = util.myConnection();
+}
+catch(Exception e)
+{
+ String theError = "ERROR: unable to get a connection";
+ throw e;
+}
+
+sessionUser = util.getSessionUser(conn);
+currentUser = util.getCurrentUser(conn);
+currentTime = util.getCurrentTime(conn);
+
+results[0] = "External user: " + sessionUser + '\n';
+results[0] += "Current user: " + currentUser + '\n';
+results[0] += "Current time: " + currentTime + '\n';
+
+util.log("Session user: " + sessionUser);
+util.log("Current user: " + currentUser);
+util.log("Current time: " + currentTime);
+
+try
+{
+ int numRows;
+ String selectQuery = "select count(*) from hive.hive.";
+ selectQuery += operationIn;
+ Statement stmt = conn.createStatement();
+ util.log("Executing select query");
+ ResultSet rset = stmt.executeQuery(selectQuery);
+ if (rset.next())
+ {
+numRows = rset.getInt(1);
+results[0] +
