usergrid git commit: USERGRID-1044: add audit delete test (delete only, no verify)

[mdunker]

Repository: usergrid
Updated Branches:
  refs/heads/USERGRID-1044 0fc18041e -> 247f79603


USERGRID-1044: add audit delete test (delete only, no verify)


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/247f7960
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/247f7960
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/247f7960

Branch: refs/heads/USERGRID-1044
Commit: 247f79603d1775273a56274ce7810f1b8f1f5d29
Parents: 0fc1804
Author: Mike Dunker 
Authored: Mon Nov 2 07:34:42 2015 -0800
Committer: Mike Dunker 
Committed: Mon Nov 2 07:34:42 2015 -0800

--
 stack/loadtests/runAuditDeleteEntities.sh   | 65 
 .../runAuditVerifyCollectionEntities.sh |  2 +-
 .../apache/usergrid/enums/ScenarioType.scala|  3 +-
 .../usergrid/scenarios/AuditScenarios.scala | 49 ++-
 .../usergrid/simulations/AuditSimulation.scala  |  4 +-
 5 files changed, 118 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/247f7960/stack/loadtests/runAuditDeleteEntities.sh
--
diff --git a/stack/loadtests/runAuditDeleteEntities.sh 
b/stack/loadtests/runAuditDeleteEntities.sh
new file mode 100755
index 000..ff92940
--- /dev/null
+++ b/stack/loadtests/runAuditDeleteEntities.sh
@@ -0,0 +1,65 @@
+#!/bin/bash
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+DIR="${BASH_SOURCE%/*}"
+if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
+. "$DIR/testConfig.sh"
+
+# from testConfig.sh
+#URL=
+#ADMIN_USER=
+#ADMIN_PASSWORD=
+#ORG=
+#APP=
+#AUTH_TYPE=
+#TOKEN_TYPE=
+#RETRY_COUNT=
+
+die() { echo "$@" 1>&2 ; exit 1; }
+
+[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [FAILED_UUID_FILENAME 
[USERGRID_REGION]]"
+
+RAMP_USERS="$1"
+RAMP_TIME="$2"
+AUDIT_UUID_FILENAME="$3"
+FAILED_UUID_FILENAME="$4"
+[ "$#" -ge 5 ] && USERGRID_REGION="$5"
+
+shift $#
+
+SCENARIO_TYPE=auditDeleteEntities
+
+#Compile everything
+mvn compile
+
+#Execute the test
+mvn gatling:execute \
+-DbaseUrl=${URL} \
+-DadminUser=${ADMIN_USER}  \
+-DadminPassword=${ADMIN_PASSWORD}  \
+-Dorg=${ORG} \
+-Dapp=${APP} \
+-DauthType=${AUTH_TYPE} \
+-DtokenType=${TOKEN_TYPE} \
+-DretryCount=${RETRY_COUNT} \
+-DscenarioType=${SCENARIO_TYPE} \
+-DrampUsers=${RAMP_USERS}  \
+-DrampTime=${RAMP_TIME}  \
+-DauditUuidFilename=${AUDIT_UUID_FILENAME} \
+-DfailedUuidFilename=${FAILED_UUID_FILENAME} \
+-DprintFailedRequests=${PRINT_FAILED_REQUESTS} \
+-DusergridRegion=${USERGRID_REGION} \
+-Dgatling.simulationClass=org.apache.usergrid.simulations.AuditSimulation
+

http://git-wip-us.apache.org/repos/asf/usergrid/blob/247f7960/stack/loadtests/runAuditVerifyCollectionEntities.sh
--
diff --git a/stack/loadtests/runAuditVerifyCollectionEntities.sh 
b/stack/loadtests/runAuditVerifyCollectionEntities.sh
index 695c9ee..be50189 100755
--- a/stack/loadtests/runAuditVerifyCollectionEntities.sh
+++ b/stack/loadtests/runAuditVerifyCollectionEntities.sh
@@ -31,7 +31,7 @@ DELETE_AFTER_SUCCESSFUL_AUDIT=false
 
 die() { echo "$@" 1>&2 ; exit 1; }
 
-[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [FAILED_UUID_FILENAME 
[DELETE_AFTER_SUCCESSFUL_AUDIT(true/false)[USERGRID_REGION]]]"
+[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [FAILED_UUID_FILENAME 
[DELETE_AFTER_SUCCESSFUL_AUDIT(true/false) [USERGRID_REGION]]]"
 
 RAMP_USERS="$1"
 RAMP_TIME="$2"

http://git-wip-us.apache.org/repos/asf/usergrid/blob/247f7960/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala 
b/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala
index c86dc33..1cc1fa2 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala
+++ 

[01/17] usergrid git commit: USERGRID-1044: set default retry count to 0 also, retry count was being used as the total number of attempts

[mrusso]

Repository: usergrid
Updated Branches:
  refs/heads/master b031d4d3d -> 568d35d57


USERGRID-1044: set default retry count to 0
also, retry count was being used as the total number of attempts


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/1195ce85
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/1195ce85
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/1195ce85

Branch: refs/heads/master
Commit: 1195ce85fefcc8334b38f6e79cc8a2c53ce9ce10
Parents: 2f307a7
Author: Mike Dunker 
Authored: Tue Oct 20 13:37:34 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 20 13:37:34 2015 -0700

--
 stack/loadtests/README.md |  2 +-
 .../org/apache/usergrid/enums/ConfigProperties.scala  |  2 +-
 .../apache/usergrid/scenarios/AuditScenarios.scala|  6 +++---
 .../scenarios/EntityCollectionScenarios.scala | 14 +++---
 stack/loadtests/testConfig.sh |  4 ++--
 5 files changed, 14 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/1195ce85/stack/loadtests/README.md
--
diff --git a/stack/loadtests/README.md b/stack/loadtests/README.md
index 802c286..ec34b42 100644
--- a/stack/loadtests/README.md
+++ b/stack/loadtests/README.md
@@ -106,7 +106,7 @@ Defaults listed are those that are specified by the 
Usergrid Gatling code, not n
 * uuidFilename (no default) - UUID filename for non-audit tests
 * auditUuidFilename (no default) - UUID filename for audit tests
 * failedUuidFilename (no default) - UUID filename for failed entries while 
auditing
-* retryCount (**5**) - number of retries of operation before giving up
+* retryCount (**0**) - number of retries of operation before giving up
 * purgeUsers (**100**) - number of entities to create at a time during 
loadEntities (send this many requests, then wait for the responses)
 * laterThanTimestamp (**0**) - if specified for an audit, will only match 
entities modified >= this timestamp; can be used for incremental audits
 * entityProgressCount (**1**) - print to console every time this many 
requests has been sent (if entityProgressCount = 1, will print on the 
1000th, 2000th, 3000th... request)

http://git-wip-us.apache.org/repos/asf/usergrid/blob/1195ce85/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ConfigProperties.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ConfigProperties.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ConfigProperties.scala
index f74222e..cbb484c 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ConfigProperties.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ConfigProperties.scala
@@ -153,7 +153,7 @@ object ConfigProperties {
 case FailedUuidFilename => "/tmp/dummyFailedUuid.csv"
 case SandboxCollection => false
 case PurgeUsers => 100
-case RetryCount => 5
+case RetryCount => 0
 case LaterThanTimestamp => 0L
 case EntityProgressCount => 1L
 case InjectionList => "rampUsers(10,60)"

http://git-wip-us.apache.org/repos/asf/usergrid/blob/1195ce85/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
index 4743536..4477338 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
@@ -100,10 +100,10 @@ object AuditScenarios {
   session
 }
 .doIf(session => session("validEntity").as[String] == "yes") {
-  tryMax(Settings.retryCount) {
+  tryMax(1+Settings.retryCount) {
 exec(getCollectionsWithoutCursor)
   }.asLongAs(stringParamExists(SessionVarCursor)) {
-tryMax(Settings.retryCount) {
+tryMax(1+Settings.retryCount) {
   exec(getCollectionsWithCursor)
 }
   }
@@ -252,7 +252,7 @@ object AuditScenarios {
 .asLongAs(session => 
session("validEntity").asOption[String].map(validEntity => validEntity != 
"no").getOrElse[Boolean](true)) {
 feed(FeederGenerator.collectionCsvFeeder)
   .doIf(session => session("validEntity").as[String] == "yes") {
-tryMax(Settings.retryCount) {
+tryMax(1+Settings.retryCount) {
   

[15/17] usergrid git commit: USERGRID-1044: add audit delete test (delete only, no verify)

[mrusso]

USERGRID-1044: add audit delete test (delete only, no verify)


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/247f7960
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/247f7960
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/247f7960

Branch: refs/heads/master
Commit: 247f79603d1775273a56274ce7810f1b8f1f5d29
Parents: 0fc1804
Author: Mike Dunker 
Authored: Mon Nov 2 07:34:42 2015 -0800
Committer: Mike Dunker 
Committed: Mon Nov 2 07:34:42 2015 -0800

--
 stack/loadtests/runAuditDeleteEntities.sh   | 65 
 .../runAuditVerifyCollectionEntities.sh |  2 +-
 .../apache/usergrid/enums/ScenarioType.scala|  3 +-
 .../usergrid/scenarios/AuditScenarios.scala | 49 ++-
 .../usergrid/simulations/AuditSimulation.scala  |  4 +-
 5 files changed, 118 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/247f7960/stack/loadtests/runAuditDeleteEntities.sh
--
diff --git a/stack/loadtests/runAuditDeleteEntities.sh 
b/stack/loadtests/runAuditDeleteEntities.sh
new file mode 100755
index 000..ff92940
--- /dev/null
+++ b/stack/loadtests/runAuditDeleteEntities.sh
@@ -0,0 +1,65 @@
+#!/bin/bash
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+DIR="${BASH_SOURCE%/*}"
+if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
+. "$DIR/testConfig.sh"
+
+# from testConfig.sh
+#URL=
+#ADMIN_USER=
+#ADMIN_PASSWORD=
+#ORG=
+#APP=
+#AUTH_TYPE=
+#TOKEN_TYPE=
+#RETRY_COUNT=
+
+die() { echo "$@" 1>&2 ; exit 1; }
+
+[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [FAILED_UUID_FILENAME 
[USERGRID_REGION]]"
+
+RAMP_USERS="$1"
+RAMP_TIME="$2"
+AUDIT_UUID_FILENAME="$3"
+FAILED_UUID_FILENAME="$4"
+[ "$#" -ge 5 ] && USERGRID_REGION="$5"
+
+shift $#
+
+SCENARIO_TYPE=auditDeleteEntities
+
+#Compile everything
+mvn compile
+
+#Execute the test
+mvn gatling:execute \
+-DbaseUrl=${URL} \
+-DadminUser=${ADMIN_USER}  \
+-DadminPassword=${ADMIN_PASSWORD}  \
+-Dorg=${ORG} \
+-Dapp=${APP} \
+-DauthType=${AUTH_TYPE} \
+-DtokenType=${TOKEN_TYPE} \
+-DretryCount=${RETRY_COUNT} \
+-DscenarioType=${SCENARIO_TYPE} \
+-DrampUsers=${RAMP_USERS}  \
+-DrampTime=${RAMP_TIME}  \
+-DauditUuidFilename=${AUDIT_UUID_FILENAME} \
+-DfailedUuidFilename=${FAILED_UUID_FILENAME} \
+-DprintFailedRequests=${PRINT_FAILED_REQUESTS} \
+-DusergridRegion=${USERGRID_REGION} \
+-Dgatling.simulationClass=org.apache.usergrid.simulations.AuditSimulation
+

http://git-wip-us.apache.org/repos/asf/usergrid/blob/247f7960/stack/loadtests/runAuditVerifyCollectionEntities.sh
--
diff --git a/stack/loadtests/runAuditVerifyCollectionEntities.sh 
b/stack/loadtests/runAuditVerifyCollectionEntities.sh
index 695c9ee..be50189 100755
--- a/stack/loadtests/runAuditVerifyCollectionEntities.sh
+++ b/stack/loadtests/runAuditVerifyCollectionEntities.sh
@@ -31,7 +31,7 @@ DELETE_AFTER_SUCCESSFUL_AUDIT=false
 
 die() { echo "$@" 1>&2 ; exit 1; }
 
-[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [FAILED_UUID_FILENAME 
[DELETE_AFTER_SUCCESSFUL_AUDIT(true/false)[USERGRID_REGION]]]"
+[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [FAILED_UUID_FILENAME 
[DELETE_AFTER_SUCCESSFUL_AUDIT(true/false) [USERGRID_REGION]]]"
 
 RAMP_USERS="$1"
 RAMP_TIME="$2"

http://git-wip-us.apache.org/repos/asf/usergrid/blob/247f7960/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala 
b/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala
index c86dc33..1cc1fa2 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ScenarioType.scala
@@ -26,10 +26,11 @@ object ScenarioType {
   val 

[14/17] usergrid git commit: USERGRID-1044: don't log UUID mismatch if UUID was never retrieved during load

[mrusso]

USERGRID-1044: don't log UUID mismatch if UUID was never retrieved during load


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/0fc18041
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/0fc18041
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/0fc18041

Branch: refs/heads/master
Commit: 0fc18041e32df049040f26a0faaef1f46dbe6d9f
Parents: 2a09bf1
Author: Mike Dunker 
Authored: Wed Oct 28 08:09:19 2015 -0700
Committer: Mike Dunker 
Committed: Wed Oct 28 08:09:19 2015 -0700

--
 .../scala/org/apache/usergrid/scenarios/AuditScenarios.scala | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/0fc18041/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
index d1e2c4b..cc59e0a 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
@@ -183,7 +183,7 @@ object AuditScenarios {
   errorString += "|NoUuidReturned"
   Settings.incAuditPayloadUuidError()
   println(s"PAYLOAD UUID MISSING (DIRECT): requestedUuid=$uuid")
-} else if (!uuid.equalsIgnoreCase(entityUuid)) {
+} else if (!uuid.isEmpty && !uuid.equalsIgnoreCase(entityUuid)) {
   errorString += "|ReturnedUuidMismatch"
   Settings.incAuditPayloadUuidError()
   println(s"PAYLOAD UUID MISMATCH (DIRECT): requestedUuid=$uuid 
returnedUuid=$entityUuid")
@@ -247,7 +247,7 @@ object AuditScenarios {
 errorString += "|NoUuidReturned"
 Settings.incAuditPayloadUuidError()
 println(s"PAYLOAD UUID MISSING (QUERY): requestedUuid=$uuid")
-  } else if (!uuid.equalsIgnoreCase(entityUuid)) {
+  } else if (!uuid.isEmpty && !uuid.equalsIgnoreCase(entityUuid)) {
 errorString += "|ReturnedUuidMismatch"
 Settings.incAuditPayloadUuidError()
 println(s"PAYLOAD UUID MISMATCH (QUERY): requestedUuid=$uuid 
returnedUuid=$entityUuid")

[03/17] usergrid git commit: USERGRID-1044: make load entities unlimited save invalid requests

[mrusso]

USERGRID-1044: make load entities unlimited save invalid requests


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/1a6de02a
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/1a6de02a
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/1a6de02a

Branch: refs/heads/master
Commit: 1a6de02ab89f1e5272d1902003fb07085568c069
Parents: 7271935
Author: Mike Dunker 
Authored: Mon Oct 26 12:51:38 2015 -0700
Committer: Mike Dunker 
Committed: Mon Oct 26 12:51:38 2015 -0700

--
 stack/loadtests/runLoadEntitiesUnlimited.sh | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/1a6de02a/stack/loadtests/runLoadEntitiesUnlimited.sh
--
diff --git a/stack/loadtests/runLoadEntitiesUnlimited.sh 
b/stack/loadtests/runLoadEntitiesUnlimited.sh
index 11ca0a4..c14b6b1 100755
--- a/stack/loadtests/runLoadEntitiesUnlimited.sh
+++ b/stack/loadtests/runLoadEntitiesUnlimited.sh
@@ -99,5 +99,6 @@ mvn gatling:execute \
 -DuuidFilename=${UUID_FILENAME} \
 -DprintFailedRequests=${PRINT_FAILED_REQUESTS} \
 -DusergridRegion=${USERGRID_REGION} \
+-DsaveInvalidResponse=true \
 
-Dgatling.simulationClass=org.apache.usergrid.simulations.ConfigurableSimulation
 

[11/17] usergrid git commit: USERGRID-1044: for org/app/collection create, default to testConfig settings

[mrusso]

USERGRID-1044: for org/app/collection create, default to testConfig settings


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/fc23b79e
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/fc23b79e
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/fc23b79e

Branch: refs/heads/master
Commit: fc23b79e9a8a08bc57d25e3e60a9718304f8f1c0
Parents: 0389744
Author: Mike Dunker 
Authored: Tue Oct 27 16:39:33 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 16:39:33 2015 -0700

--
 stack/loadtests/runOrgAppSetup.sh | 17 ++---
 1 file changed, 10 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/fc23b79e/stack/loadtests/runOrgAppSetup.sh
--
diff --git a/stack/loadtests/runOrgAppSetup.sh 
b/stack/loadtests/runOrgAppSetup.sh
index 19936b7..b26eb37 100755
--- a/stack/loadtests/runOrgAppSetup.sh
+++ b/stack/loadtests/runOrgAppSetup.sh
@@ -29,6 +29,9 @@ if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 #ENTITY_TYPE=
 #ENTITY_PREFIX=
 #ENTITY_SEED=  #may be overridden on command line
+#ORG= #may be overridden on command line
+#APP= #may be overridden on command line
+#COLLECTION= #may be overridden on command line
 #RETRY_COUNT=
 #ENTITY_PROGRESS_COUNT=
 #CONSTANT_USERS_PER_SEC=
@@ -36,14 +39,14 @@ if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 
 die() { echo "$@" 1>&2 ; exit 1; }
 
-[ "$#" -ge 2 ] || die "At least 2 arguments required, $# provided.  Example is 
$0 ORG APP [COLLECTION [SANDBOX_COLLECTION (true/false)]]"
+[ "$#" -ge 1 ] || die "At least 1 argument required, $# provided.  Example is 
$0 SANDBOX_COLLECTION(true/false) [ORG [APP [COLLECTION]]]"
 
-ORG="$1"
-APP="$2"
-COLLECTION="gatlingitems"
-[ "$#" -ge 3 ] && COLLECTION="$3"
-SANDBOX_COLLECTION=true
-[ "$#" -ge 4 ] && SANDBOX_COLLECTION="$4"
+
+SANDBOX_COLLECTION="$1"
+# org, app, and collection can come from testConfig.sh
+[ "$#" -ge 2 ] && ORG="$2"
+[ "$#" -ge 3 ] && APP="$3"
+[ "$#" -ge 4 ] && COLLECTION="$4"
 
 shift $#
 

[10/17] usergrid git commit: USERGRID-1044: print non-200s during load (including body)

[mrusso]

USERGRID-1044: print non-200s during load (including body)


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/03897447
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/03897447
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/03897447

Branch: refs/heads/master
Commit: 0389744752f479d215f84ebbd5f25aa36d2108a4
Parents: 83d7a6f
Author: Mike Dunker 
Authored: Tue Oct 27 14:59:10 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 14:59:10 2015 -0700

--
 .../usergrid/scenarios/EntityCollectionScenarios.scala   | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/03897447/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
index f37a2d2..d5b3101 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
@@ -42,6 +42,7 @@ object EntityCollectionScenarios {
   val SessionVarStatus: String = "status"
   val SessionVarUuid: String = "createUuid"
   val SessionVarModified: String = "createModified"
+  val SessionVarBodyString: String = "bodyString"
 
   def entityGetUrl(useCursor: Boolean): String = {
 val url = s"/${Settings.collection}?" +
@@ -225,11 +226,12 @@ object EntityCollectionScenarios {
 new ResponseWrapper(response) {
   val contentType = 
response.header("content-type").getOrElse("").toLowerCase
   val bodyStr = if (contentType.contains("json")) 
response.body.string else "[]"
+  if (bodyStr == "[]") { println(">>> USING EMPTY BODY") }
   override val body = StringResponseBody(bodyStr, response.charset)
 }
 }
 // 200 for success, 400 if already exists
-.check(status.saveAs(SessionVarStatus), 
extractEntityUuid(SessionVarUuid), extractEntityModified(SessionVarModified)))
+.check(status.saveAs(SessionVarStatus), 
bodyString.saveAs(SessionVarBodyString), extractEntityUuid(SessionVarUuid), 
extractEntityModified(SessionVarModified)))
 .exec(session => {
   val saveFailures = Settings.saveInvalidResponse
   val status = session(SessionVarStatus).as[Int]
@@ -239,6 +241,10 @@ object EntityCollectionScenarios {
 val entityName = session("entityName").as[String]
 val modified = if (status == 200) 
session(SessionVarModified).as[Long] else 0
 val collectionName = session("collectionName").as[String]
+if (status != 200) {
+  val bodyString = session(SessionVarBodyString).as[String]
+  println(s" LOAD ERROR - Status: 
$status\nBody:\n$bodyString")
+}
 Settings.addUuid(uuid, collectionName, entityName, modified, 
status)
 session
   } else {

[09/17] usergrid git commit: USERGRID-1044: fixes

[mrusso]

USERGRID-1044: fixes


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/83d7a6f8
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/83d7a6f8
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/83d7a6f8

Branch: refs/heads/master
Commit: 83d7a6f83e005ee8a4794ed86f4f0de95653360c
Parents: be0033e
Author: Mike Dunker 
Authored: Tue Oct 27 13:44:33 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 13:44:33 2015 -0700

--
 .../usergrid/datagenerators/FeederGenerator.scala |  3 ++-
 .../usergrid/scenarios/AuditScenarios.scala   | 18 --
 .../org/apache/usergrid/settings/Settings.scala   | 10 +-
 3 files changed, 15 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/83d7a6f8/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
index 73b3d3c..38db19a 100755
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
@@ -279,11 +279,12 @@ object FeederGenerator {
   val name = if (line != null) array(1) else ""
   val uuid = if (line != null) array(2) else ""
   val modified = if (line != null) array(3) else ""
+  val lastStatus = if (line != null) array(4) else ""
   //println(s"$collectionName|$name|$uuid|$modified")
   val accessField = if (uuid != "") uuid else name
   val queryField = if (uuid != "") s"uuid='$uuid'" else s"name='$name'"
 
-  Map("collectionName" -> collectionName, "name" -> name,  "uuid" -> uuid, 
"modified" -> modified,
+  Map("collectionName" -> collectionName, "name" -> name,  "uuid" -> uuid, 
"modified" -> modified, "lastStatus" -> lastStatus,
 "validEntity" -> validEntity, "accessField" -> accessField, 
"queryField" -> queryField)
 }
   }

http://git-wip-us.apache.org/repos/asf/usergrid/blob/83d7a6f8/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
index 86c0b73..1ce2bc2 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
@@ -157,8 +157,7 @@ object AuditScenarios {
 val modified = session("modified").as[String].toLong
 val uuid = session("uuid").as[String]
 val reqName = session("name").as[String]
-val prevStatus = session("status").as[Int]
-val prevError = session("error").as[String]
+val lastStatus = session("lastStatus").as[String]
 if (status == 200 || status == 404) {
   val collectionEntities = 
session(SessionVarCollectionEntities).as[Seq[Any]]
   val entityUuid = session(SessionVarEntityUuid).as[String]
@@ -166,12 +165,12 @@ object AuditScenarios {
 
   val count = collectionEntities.length
   if (count < 1) {
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"NotFoundAtAll", prevStatus, prevError)
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"NotFoundAtAll", lastStatus)
 Settings.incAuditNotFoundAtAll()
 println(s"NOT FOUND AT ALL: $collectionName.$reqName ($uuid)")
   } else if (count > 1) {
 // invalid
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"QueryInvalidCount$count", prevStatus, prevError)
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"QueryInvalidCount$count", lastStatus)
 Settings.incAuditBadResponse()
 println(s"INVALID RESPONSE (count=$count): 
$collectionName.$reqName ($uuid)")
   } else {
@@ -199,13 +198,13 @@ object AuditScenarios {
   println(s"PAYLOAD NAME MISMATCH (DIRECT): requestedName=$reqName 
returnedName=$entityName")
 }
 
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, errorString, prevStatus, prevError)
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, errorString, lastStatus)
 Settings.incAuditNotFoundViaQuery()

[06/17] usergrid git commit: USERGRID-1044: attempt #2 to preprocess response body

[mrusso]

USERGRID-1044: attempt #2 to preprocess response body


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/4b2d0035
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/4b2d0035
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/4b2d0035

Branch: refs/heads/master
Commit: 4b2d00355e5645ab5c158b4c4f8e228037879af4
Parents: 2778f0a
Author: Mike Dunker 
Authored: Tue Oct 27 11:29:16 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 11:29:16 2015 -0700

--
 .../usergrid/scenarios/EntityCollectionScenarios.scala | 13 -
 1 file changed, 4 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/4b2d0035/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
index bcdb49a..f37a2d2 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
@@ -223,15 +223,10 @@ object EntityCollectionScenarios {
 .transformResponse {
   case response if response.isReceived =>
 new ResponseWrapper(response) {
-  override val body = {
-val contentType = 
response.header("content-type").getOrElse("").toLowerCase
-if (contentType.contains("json")) {
-  StringResponseBody(response.body.string, response.charset)
-} else {
-  StringResponseBody("{}", StandardCharsets.UTF_8)
-}
-  }
-  }
+  val contentType = 
response.header("content-type").getOrElse("").toLowerCase
+  val bodyStr = if (contentType.contains("json")) 
response.body.string else "[]"
+  override val body = StringResponseBody(bodyStr, response.charset)
+}
 }
 // 200 for success, 400 if already exists
 .check(status.saveAs(SessionVarStatus), 
extractEntityUuid(SessionVarUuid), extractEntityModified(SessionVarModified)))

usergrid git commit: Removed commented out older code.

[grey]

Repository: usergrid
Updated Branches:
  refs/heads/userCollectionFix a40cf2843 -> 0b3eea179


Removed commented out older code.


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/0b3eea17
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/0b3eea17
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/0b3eea17

Branch: refs/heads/userCollectionFix
Commit: 0b3eea1793405a089857d3c14cc898b6a83cd41a
Parents: a40cf28
Author: George Reyes 
Authored: Mon Nov 2 09:38:32 2015 -0800
Committer: George Reyes 
Committed: Mon Nov 2 09:38:32 2015 -0800

--
 .../usergrid/tools/CollectionUserFix.java   | 127 +--
 1 file changed, 4 insertions(+), 123 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/0b3eea17/stack/tools/src/main/java/org/apache/usergrid/tools/CollectionUserFix.java
--
diff --git 
a/stack/tools/src/main/java/org/apache/usergrid/tools/CollectionUserFix.java 
b/stack/tools/src/main/java/org/apache/usergrid/tools/CollectionUserFix.java
index e63bca8..9b95c7a 100644
--- a/stack/tools/src/main/java/org/apache/usergrid/tools/CollectionUserFix.java
+++ b/stack/tools/src/main/java/org/apache/usergrid/tools/CollectionUserFix.java
@@ -104,15 +104,6 @@ public class CollectionUserFix extends ExportingToolBase {
 
 
 /**
- * psudeo code
- * if org id is present then ignore the application id
- *  go through everysingle application/users entity. And check to see 
if it can be queried by username.
- * else if app id is present
- *  go through the applications/users
- *
- *
- * Take the list of applications to go through and go through them one by 
one. ( in the latter else case we will only
- *
  * @param line
  * @throws Exception
  */
@@ -149,8 +140,6 @@ public class CollectionUserFix extends ExportingToolBase {
 System.out.println("Repair Complete");
 }
 
-
-//make
 private void startCollectionFlow(final EntityManager entityManager, final 
Set app, final String queryString )
 throws Exception {// search for all orgs
 
@@ -159,51 +148,21 @@ public class CollectionUserFix extends ExportingToolBase {
 query = query.fromQL( queryString );
 }
 query.setLimit( PAGE_SIZE );
-Results r = null;
 EntityManager em = null;
-int numberOfUsers = 0;
 Identifier identifier = new Identifier();
 
 
 for ( Application application : app ) {
-//This will hold all of the applications users. This will be 
stored in memory to do a simple check to see if
-//there are any duped usernames in the collection.
-//Memory concerns means that
 
-//This means that we need to set it for each and every single 
application thus it gets set here instead of
-//the method that calls us.
-if(entityManager == null){
+if(entityManager == null || !em.getApplication().equals( 
application )){
 em = emf.getEntityManager( application.getUuid() );
 }
 else {
 em = entityManager;
 }
-//
-//do {
-//Multimap usernames = HashMultimap.create();
-//
-//
-////get all users in the management app and page for each set 
of a PAGE_SIZE
-//r = em.searchCollection( application, "users", query );
-//numberOfUsers+=r.size();
-//System.out.println("found "+numberOfUsers+" users");
-//
-//for ( Entity entity : r.getEntities() ) {
-////grab all usernames returned.
-//usernames.put( entity.getProperty( "username" 
).toString().toLowerCase(), entity.getUuid() );
-//}
-//
-//query.setCursor( r.getCursor() );
-//
-//System.out.println("Starting username crawl of 
"+usernames.size()+" number of usernames");
-//usernameVerificationFix( em, usernames );
-//
-//
-//}
-//while ( r != null && r.size() == PAGE_SIZE);
-
-r = em.searchCollection( application, "users", query );
-PagingResultsIterator pagingResultsIterator = new 
PagingResultsIterator( r );
+
+PagingResultsIterator pagingResultsIterator =
+new PagingResultsIterator( em.searchCollection( 
application, "users", query ) );
 
 while(pagingResultsIterator.hasNext()){
 Entity entity = ( Entity ) pagingResultsIterator.next();
@@ -211,86 +170,8 @@ public class CollectionUserFix extends ExportingToolBase {
 em.getUserByIdentifier( 

usergrid git commit: USERGRID-1044: update README

[mdunker]

Repository: usergrid
Updated Branches:
  refs/heads/USERGRID-1044 247f79603 -> 73dcaebd2


USERGRID-1044: update README


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/73dcaebd
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/73dcaebd
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/73dcaebd

Branch: refs/heads/USERGRID-1044
Commit: 73dcaebd286fad3edffd0a8e523b31b042b061c2
Parents: 247f796
Author: Mike Dunker 
Authored: Mon Nov 2 08:54:18 2015 -0800
Committer: Mike Dunker 
Committed: Mon Nov 2 08:54:18 2015 -0800

--
 stack/loadtests/README.md | 12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/73dcaebd/stack/loadtests/README.md
--
diff --git a/stack/loadtests/README.md b/stack/loadtests/README.md
index ec34b42..9dc24e9 100644
--- a/stack/loadtests/README.md
+++ b/stack/loadtests/README.md
@@ -9,6 +9,9 @@ The test scripts are found in the top level loadtests 
directory. Look inside the
 ###testConfig.sh
 Contains defaults that are used for all the other test scripts.
 
+###runAuditDeleteEntities.sh
+For a specified organization and given CSV file, delete all entities, writing 
those that fail to delete to another CSV file.
+
 ###runAuditGetAllAppCollectionEntities.sh
 For a specified organization, finds all apps, and for each app, finds all 
collections and writes a CSV file line containing collection name, UUID, entity 
name, and modified timestamp for each entity in each collection.
 
@@ -16,7 +19,7 @@ For a specified organization, finds all apps, and for each 
app, finds all collec
 For a specified organization and application, finds all collections and writes 
a CSV file line containing collection name, UUID, entity name, and modified 
timestamp for each entity in each collection.
 
 ###runAuditVerifyCollectionEntities.sh
-For a specified organization and given CSV file, verify that all entities are 
retrievable, writing those that fail to another CSV file.
+For a specified organization and given CSV file, verify that all entities are 
retrievable via query, writing those that fail to another CSV file, optionally 
deleting verified entities.
 
 ###runCollectionQueryTest.sh
 For a given collection, retrieve all entities using a cursor and a query.
@@ -115,6 +118,13 @@ Defaults listed are those that are specified by the 
Usergrid Gatling code, not n
 * getViaQuery (**false**) - retrieve entities via query instead of via name or 
uuid
 * queryParams (**""**) - additional query parameters (currently used for get 
by entity or by name)
 * csvFeedPattern (**"random"**) - pattern to use when feeding from a CSV 
("random" is random, "circular" goes through CSV sequentially and restarts from 
beginning when it reaches the end)
+* unlimitedFeed (**false**) - continue loading with no limit on number of 
entities (forces interleavedWorkerFeed=true); hit CTRL-c to abort
+* flushCsv (**0**) - if > 0, flush CSV file when that number of CSV entries 
has been received
+* interleavedWorkerFeed (**false**) - for multiple gatling servers, shard the 
entities via interleaving rather than splitting into separate chunks; for 
example, for 10 workers over 1M entities, interleaving would cause one worker 
to have 1, 11, 21, 31, etc. instead of 1-10 
+* newCsvOnFlush (**false**) - when the output CSV file is flushed, create a 
new CSV file
+* deleteAfterSuccessfulAudit (**false**) - delete a record that is 
successfully verified via query
+* usergridRegion (**""**) - send specified region in the UsergridRegion header 
for all requests -- this allows Edge proxy to call different regions for testing
+* saveInvalidResponse (**false**) - on audit, do not retry but immediately 
save failure to output CSV file
 
 The following settings are currently not used (were used by deprecated tests, 
but may be valid in the future):
 

[13/17] usergrid git commit: USERGRID-1044: print progress during audit and log failures as such

[mrusso]

USERGRID-1044: print progress during audit and log failures as such


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/2a09bf15
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/2a09bf15
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/2a09bf15

Branch: refs/heads/master
Commit: 2a09bf15d8063199bf25b7e7f437929601f7193a
Parents: 2e70a72
Author: Mike Dunker 
Authored: Wed Oct 28 07:28:51 2015 -0700
Committer: Mike Dunker 
Committed: Wed Oct 28 07:28:51 2015 -0700

--
 .../org/apache/usergrid/datagenerators/FeederGenerator.scala | 8 
 .../scala/org/apache/usergrid/scenarios/AuditScenarios.scala | 6 +++---
 .../usergrid/scenarios/EntityCollectionScenarios.scala   | 8 +---
 3 files changed, 16 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/2a09bf15/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
index 38db19a..a9225b2 100755
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
@@ -255,6 +255,9 @@ object FeederGenerator {
 val csvLines = if (Settings.feedAuditUuids) 
Source.fromFile(Settings.feedAuditUuidFilename).getLines().toArray else 
Array[String]()
 val csvLinesLen = csvLines.length
 var counter = new AtomicInteger(0)
+var entityCounter: Long = 0L
+var lastEntityCountPrinted: Long = 0L
+var entityProgressCount: Long = Settings.entityProgressCount
 
 override def hasNext: Boolean = true
 
@@ -284,6 +287,11 @@ object FeederGenerator {
   val accessField = if (uuid != "") uuid else name
   val queryField = if (uuid != "") s"uuid='$uuid'" else s"name='$name'"
 
+  if (entityProgressCount > 0L && entityCounter >= lastEntityCountPrinted 
+ entityProgressCount) {
+println(s"CSV Entity: $entityCounter")
+lastEntityCountPrinted = entityCounter
+  }
+
   Map("collectionName" -> collectionName, "name" -> name,  "uuid" -> uuid, 
"modified" -> modified, "lastStatus" -> lastStatus,
 "validEntity" -> validEntity, "accessField" -> accessField, 
"queryField" -> queryField)
 }

http://git-wip-us.apache.org/repos/asf/usergrid/blob/2a09bf15/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
index 1ce2bc2..d1e2c4b 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
@@ -202,15 +202,15 @@ object AuditScenarios {
 Settings.incAuditNotFoundViaQuery()
 println(s"NOT FOUND VIA QUERY: $collectionName.$reqName ($uuid)")
   }
-  session
+  session.markAsFailed
 } else if (saveFailures) {
   Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, "Failure", lastStatus)
-  session
+  session.markAsFailed
 } else {
   session.markAsFailed
 }
 
-  }).exitHereIfFailed
+  })
 
   val getCollectionEntity = exec(
 http("GET collection entity")

http://git-wip-us.apache.org/repos/asf/usergrid/blob/2a09bf15/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
index d5b3101..22b9f08 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/EntityCollectionScenarios.scala
@@ -241,16 +241,18 @@ object EntityCollectionScenarios {
 val entityName = session("entityName").as[String]
 val modified = if (status == 200) 
session(SessionVarModified).as[Long] else 0
 val collectionName = session("collectionName").as[String]
+Settings.addUuid(uuid, collectionName, entityName, modified, 
status)
 if (status 

[16/17] usergrid git commit: USERGRID-1044: update README

[mrusso]

USERGRID-1044: update README


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/73dcaebd
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/73dcaebd
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/73dcaebd

Branch: refs/heads/master
Commit: 73dcaebd286fad3edffd0a8e523b31b042b061c2
Parents: 247f796
Author: Mike Dunker 
Authored: Mon Nov 2 08:54:18 2015 -0800
Committer: Mike Dunker 
Committed: Mon Nov 2 08:54:18 2015 -0800

--
 stack/loadtests/README.md | 12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/73dcaebd/stack/loadtests/README.md
--
diff --git a/stack/loadtests/README.md b/stack/loadtests/README.md
index ec34b42..9dc24e9 100644
--- a/stack/loadtests/README.md
+++ b/stack/loadtests/README.md
@@ -9,6 +9,9 @@ The test scripts are found in the top level loadtests 
directory. Look inside the
 ###testConfig.sh
 Contains defaults that are used for all the other test scripts.
 
+###runAuditDeleteEntities.sh
+For a specified organization and given CSV file, delete all entities, writing 
those that fail to delete to another CSV file.
+
 ###runAuditGetAllAppCollectionEntities.sh
 For a specified organization, finds all apps, and for each app, finds all 
collections and writes a CSV file line containing collection name, UUID, entity 
name, and modified timestamp for each entity in each collection.
 
@@ -16,7 +19,7 @@ For a specified organization, finds all apps, and for each 
app, finds all collec
 For a specified organization and application, finds all collections and writes 
a CSV file line containing collection name, UUID, entity name, and modified 
timestamp for each entity in each collection.
 
 ###runAuditVerifyCollectionEntities.sh
-For a specified organization and given CSV file, verify that all entities are 
retrievable, writing those that fail to another CSV file.
+For a specified organization and given CSV file, verify that all entities are 
retrievable via query, writing those that fail to another CSV file, optionally 
deleting verified entities.
 
 ###runCollectionQueryTest.sh
 For a given collection, retrieve all entities using a cursor and a query.
@@ -115,6 +118,13 @@ Defaults listed are those that are specified by the 
Usergrid Gatling code, not n
 * getViaQuery (**false**) - retrieve entities via query instead of via name or 
uuid
 * queryParams (**""**) - additional query parameters (currently used for get 
by entity or by name)
 * csvFeedPattern (**"random"**) - pattern to use when feeding from a CSV 
("random" is random, "circular" goes through CSV sequentially and restarts from 
beginning when it reaches the end)
+* unlimitedFeed (**false**) - continue loading with no limit on number of 
entities (forces interleavedWorkerFeed=true); hit CTRL-c to abort
+* flushCsv (**0**) - if > 0, flush CSV file when that number of CSV entries 
has been received
+* interleavedWorkerFeed (**false**) - for multiple gatling servers, shard the 
entities via interleaving rather than splitting into separate chunks; for 
example, for 10 workers over 1M entities, interleaving would cause one worker 
to have 1, 11, 21, 31, etc. instead of 1-10 
+* newCsvOnFlush (**false**) - when the output CSV file is flushed, create a 
new CSV file
+* deleteAfterSuccessfulAudit (**false**) - delete a record that is 
successfully verified via query
+* usergridRegion (**""**) - send specified region in the UsergridRegion header 
for all requests -- this allows Edge proxy to call different regions for testing
+* saveInvalidResponse (**false**) - on audit, do not retry but immediately 
save failure to output CSV file
 
 The following settings are currently not used (were used by deprecated tests, 
but may be valid in the future):
 

[04/17] usergrid git commit: USERGRID-1044: make jsonPath extraction optional to keep users from exiting

[mrusso]

USERGRID-1044: make jsonPath extraction optional to keep users from exiting


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/3c60c49a
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/3c60c49a
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/3c60c49a

Branch: refs/heads/master
Commit: 3c60c49a157be63ec626ff88666472f2d4876554
Parents: 1a6de02
Author: Mike Dunker 
Authored: Tue Oct 27 08:52:17 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 08:52:17 2015 -0700

--
 .../org/apache/usergrid/helpers/Extractors.scala  | 18 +++---
 1 file changed, 11 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/3c60c49a/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala 
b/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
index 136a561..b21f9d7 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
@@ -37,7 +37,7 @@ object Extractors {
 jsonPath("$.cursor").transformOption(extract => {
   //it may or may not be present.  If it is, save it, otherwise save it as 
an empty string
   extract.orElse(Some(""))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -47,7 +47,7 @@ object Extractors {
 jsonPath("$.entities[0].uuid").transformOption(extract => {
   //it may or may not be present.  If it is, save it, otherwise save it as 
an empty string
   extract.orElse(Some(""))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -57,7 +57,7 @@ object Extractors {
 jsonPath("$.entities[0].name").transformOption(extract => {
   //it may or may not be present.  If it is, save it, otherwise save it as 
an empty string
   extract.orElse(Some(""))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -67,21 +67,25 @@ object Extractors {
 jsonPath("$.entities[0].modified").ofType[Long].transformOption(extract => 
{
   //it may or may not be present.  If it is, save it, otherwise save it as 
-1
   extract.orElse(Some(-1))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
* Will extract the audit entities from the get collection response.
*/
   def extractAuditEntities(saveAsName: String) = {
-
jsonPath("$.entities[*]").ofType[Map[String,Any]].findAll.transformOption(extract
 => { extract.orElse(Some(Seq.empty)) }).saveAs(saveAsName)
+
jsonPath("$.entities[*]").ofType[Map[String,Any]].findAll.transformOption(extract
 => {
+  extract.orElse(Some(Seq.empty))
+}).optional.saveAs(saveAsName)
   }
 
   /**
* Will extract the audit entities from the get collection response.
*/
   def extractAuditEntity(saveAsName: String) = {
-
jsonPath("$.entities[0]").ofType[Map[String,Any]].findAll.transformOption(extract
 => { extract.orElse(Some(Seq.empty)) }).saveAs(saveAsName)
+
jsonPath("$.entities[0]").ofType[Map[String,Any]].findAll.transformOption(extract
 => {
+  extract.orElse(Some(Seq.empty))
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -100,7 +104,7 @@ object Extractors {
   def maybeExtractEntities(saveAsName: String) = {
 jsonPath("$.entities").ofType[Seq[Any]].transformOption(extract => {
   extract.orElse(Some(Seq()))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**

[12/17] usergrid git commit: USERGRID-1044: add shutdown handler to audit simulations

[mrusso]

USERGRID-1044: add shutdown handler to audit simulations


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/2e70a72b
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/2e70a72b
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/2e70a72b

Branch: refs/heads/master
Commit: 2e70a72b99ac0a48e5f703a1b464fdad3c4d2665
Parents: fc23b79
Author: Mike Dunker 
Authored: Tue Oct 27 16:45:02 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 16:45:02 2015 -0700

--
 .../apache/usergrid/simulations/AuditSimulation.scala| 11 +++
 .../usergrid/simulations/ConfigurableSimulation.scala|  2 +-
 2 files changed, 12 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/2e70a72b/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/AuditSimulation.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/AuditSimulation.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/AuditSimulation.scala
index 47323ea..bdec3dd 100755
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/AuditSimulation.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/AuditSimulation.scala
@@ -57,6 +57,10 @@ class AuditSimulation extends Simulation {
   }
 
   after {
+endHandler
+  }
+
+  def endHandler: Unit = {
 Settings.setTestEndTime()
 if (Settings.captureAuditUuids) {
   val uuidDesc = Settings.scenarioType match {
@@ -69,5 +73,12 @@ class AuditSimulation extends Simulation {
 Settings.printAuditResults()
   }
 
+  def abortedEarly: Unit = {
+println("AUDIT ABORTED")
+endHandler
+  }
+
+  sys addShutdownHook abortedEarly
+
 }
 

http://git-wip-us.apache.org/repos/asf/usergrid/blob/2e70a72b/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/ConfigurableSimulation.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/ConfigurableSimulation.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/ConfigurableSimulation.scala
index 9d4b05d..a3eb595 100755
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/ConfigurableSimulation.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/simulations/ConfigurableSimulation.scala
@@ -99,7 +99,7 @@ class ConfigurableSimulation extends Simulation {
 Settings.printSettingsSummary(true)
   }
 
-  sys addShutdownHook(endHandler)
+  sys addShutdownHook endHandler
 
 }
 

[02/17] usergrid git commit: USERGRID-1044: save status code and error message for audit

[mrusso]

USERGRID-1044: save status code and error message for audit


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/7271935b
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/7271935b
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/7271935b

Branch: refs/heads/master
Commit: 7271935bb340a99f8961a99ca6f2d11b9e48b312
Parents: 1195ce8
Author: Mike Dunker 
Authored: Mon Oct 26 09:56:28 2015 -0700
Committer: Mike Dunker 
Committed: Mon Oct 26 09:56:28 2015 -0700

--
 .../runAuditGetAllAppCollectionEntities.sh  |   6 +-
 .../loadtests/runAuditGetCollectionEntities.sh  |   6 +-
 .../datagenerators/FeederGenerator.scala|   2 +-
 .../usergrid/enums/ConfigProperties.scala   |   6 +-
 .../usergrid/scenarios/AuditScenarios.scala | 144 ---
 .../scenarios/EntityCollectionScenarios.scala   |  74 ++
 .../org/apache/usergrid/settings/Settings.scala |  40 --
 7 files changed, 173 insertions(+), 105 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/7271935b/stack/loadtests/runAuditGetAllAppCollectionEntities.sh
--
diff --git a/stack/loadtests/runAuditGetAllAppCollectionEntities.sh 
b/stack/loadtests/runAuditGetAllAppCollectionEntities.sh
index c8a30a3..f1304c0 100755
--- a/stack/loadtests/runAuditGetAllAppCollectionEntities.sh
+++ b/stack/loadtests/runAuditGetAllAppCollectionEntities.sh
@@ -31,10 +31,10 @@ if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 
 die() { echo "$@" 1>&2 ; exit 1; }
 
-[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [LATER_THAN_TIMESTAMP(ms)]"
+[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) UUID_FILENAME [LATER_THAN_TIMESTAMP(ms)]"
 RAMP_USERS="$1"
 RAMP_TIME="$2"
-AUDIT_UUID_FILENAME="$3"
+UUID_FILENAME="$3"
 [ "$#" -ge 4 ] && LATER_THAN_TIMESTAMP="$4"
 
 shift 3
@@ -61,7 +61,7 @@ mvn gatling:execute \
 -DscenarioType=${SCENARIO_TYPE} \
 -DrampUsers=${RAMP_USERS}  \
 -DrampTime=${RAMP_TIME}  \
--DauditUuidFilename=${AUDIT_UUID_FILENAME} \
+-DuuidFilename=${UUID_FILENAME} \
 -DprintFailedRequests=${PRINT_FAILED_REQUESTS} \
 -Dgatling.simulationClass=org.apache.usergrid.simulations.AuditSimulation
 

http://git-wip-us.apache.org/repos/asf/usergrid/blob/7271935b/stack/loadtests/runAuditGetCollectionEntities.sh
--
diff --git a/stack/loadtests/runAuditGetCollectionEntities.sh 
b/stack/loadtests/runAuditGetCollectionEntities.sh
index 2ddc62b..7c87e25 100755
--- a/stack/loadtests/runAuditGetCollectionEntities.sh
+++ b/stack/loadtests/runAuditGetCollectionEntities.sh
@@ -32,10 +32,10 @@ if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 
 die() { echo "$@" 1>&2 ; exit 1; }
 
-[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) AUDIT_UUID_FILENAME [LATER_THAN_TIMESTAMP(ms)]"
+[ "$#" -ge 3 ] || die "At least 3 arguments required, $# provided.  Example is 
$0 RAMP_USERS RAMP_TIME(seconds) UUID_FILENAME [LATER_THAN_TIMESTAMP(ms)]"
 RAMP_USERS="$1"
 RAMP_TIME="$2"
-AUDIT_UUID_FILENAME="$3"
+UUID_FILENAME="$3"
 [ "$#" -ge 4 ] && LATER_THAN_TIMESTAMP="$4"
 
 shift 3
@@ -61,7 +61,7 @@ mvn gatling:execute \
 -DscenarioType=${SCENARIO_TYPE} \
 -DrampUsers=${RAMP_USERS}  \
 -DrampTime=${RAMP_TIME}  \
--DauditUuidFilename=${AUDIT_UUID_FILENAME} \
+-DuuidFilename=${UUID_FILENAME} \
 -DprintFailedRequests=${PRINT_FAILED_REQUESTS} \
 -Dgatling.simulationClass=org.apache.usergrid.simulations.AuditSimulation
 

http://git-wip-us.apache.org/repos/asf/usergrid/blob/7271935b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
index e08f8e8..b8a28d5 100755
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
@@ -264,7 +264,7 @@ object FeederGenerator {
 if (i >= csvLinesLen) return null
 
 val line = csvLines(i)
-if (line != Settings.auditUuidsHeader) return line
+if (line != Settings.uuidsHeader) return line
 
   } while (true)
 

http://git-wip-us.apache.org/repos/asf/usergrid/blob/7271935b/stack/loadtests/src/main/scala/org/apache/usergrid/enums/ConfigProperties.scala

usergrid git commit: Added entity unique to unique index cleanup.

[grey]

Repository: usergrid
Updated Branches:
  refs/heads/USERGRID-1076 [created] a4a7884b8


Added entity unique to unique index cleanup.


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/a4a7884b
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/a4a7884b
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/a4a7884b

Branch: refs/heads/USERGRID-1076
Commit: a4a7884b8c9f1b4f996dc3b51124c72eee3e9afa
Parents: a420523
Author: George Reyes 
Authored: Mon Nov 2 14:57:06 2015 -0800
Committer: George Reyes 
Committed: Mon Nov 2 14:57:06 2015 -0800

--
 .../main/java/org/apache/usergrid/tools/UniqueIndexCleanup.java | 5 +
 1 file changed, 5 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/a4a7884b/stack/tools/src/main/java/org/apache/usergrid/tools/UniqueIndexCleanup.java
--
diff --git 
a/stack/tools/src/main/java/org/apache/usergrid/tools/UniqueIndexCleanup.java 
b/stack/tools/src/main/java/org/apache/usergrid/tools/UniqueIndexCleanup.java
index 994e850..bf8f0f2 100644
--- 
a/stack/tools/src/main/java/org/apache/usergrid/tools/UniqueIndexCleanup.java
+++ 
b/stack/tools/src/main/java/org/apache/usergrid/tools/UniqueIndexCleanup.java
@@ -61,6 +61,7 @@ import static 
org.apache.usergrid.persistence.Schema.DICTIONARY_COLLECTIONS;
 import static org.apache.usergrid.persistence.Schema.getDefaultSchema;
 import static 
org.apache.usergrid.persistence.cassandra.ApplicationCF.ENTITY_INDEX;
 import static 
org.apache.usergrid.persistence.cassandra.ApplicationCF.ENTITY_INDEX_ENTRIES;
+import static 
org.apache.usergrid.persistence.cassandra.ApplicationCF.ENTITY_UNIQUE;
 import static 
org.apache.usergrid.persistence.cassandra.CassandraPersistenceUtils.addDeleteToMutator;
 import static 
org.apache.usergrid.persistence.cassandra.CassandraPersistenceUtils.key;
 import static 
org.apache.usergrid.persistence.cassandra.CassandraService.INDEX_ENTRY_LIST_COUNT;
@@ -255,6 +256,8 @@ public class UniqueIndexCleanup extends ToolBase {
 "Could not find reference 
to value '{}' for property '{}' on entity "
 +
 "{} in collection 
{}. " + " Forcing reindex", new Object[] { propValue, prop, id, collectionName 
} );
+Object key = key( applicationId, 
collectionName, prop, id );
+addDeleteToMutator( m, 
ENTITY_UNIQUE, key, timestamp, id );
 
 addDeleteToMutator( m, 
ENTITY_INDEX, rowKey, index.getName().duplicate(),
 timestamp );
@@ -263,6 +266,8 @@ public class UniqueIndexCleanup extends ToolBase {
 }
 
 if ( entries.size() > 1 ) {
+Object key = key( applicationId, 
collectionName, prop, id );
+addDeleteToMutator( m, 
ENTITY_UNIQUE, key, timestamp, id );
 logger.info(
 "Found more than 1 entity 
referencing unique index for property "
 + "'{}' "

[34/50] [abbrv] usergrid git commit: Rename new script to multitenant-migrate and restore old script.

[toddnine]

Rename new script to multitenant-migrate and restore old script.


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/8af152e6
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/8af152e6
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/8af152e6

Branch: refs/heads/USERGRID-909
Commit: 8af152e6bef74aa6259aff5a69428523da35d0bf
Parents: 57a613b
Author: Dave Johnson 
Authored: Thu Oct 29 18:00:54 2015 -0400
Committer: Dave Johnson 
Committed: Thu Oct 29 18:00:54 2015 -0400

--
 stack/scripts/migrate_entity_data.py | 401 +
 stack/scripts/multitenant_migrate.py | 703 ++
 2 files changed, 816 insertions(+), 288 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/8af152e6/stack/scripts/migrate_entity_data.py
--
diff --git a/stack/scripts/migrate_entity_data.py 
b/stack/scripts/migrate_entity_data.py
index 0edd319..9c01270 100644
--- a/stack/scripts/migrate_entity_data.py
+++ b/stack/scripts/migrate_entity_data.py
@@ -16,64 +16,30 @@
 # under the License.
 #
 #
-# To migrate multiple tenants within one cluster.
 #
-# STEP 1 - SETUP TENANT ONE TOMCAT RUNNING 2.1 NOT IN SERVICE AND INIT 
MIGRATION
+# Usage from a machine running Usergrid with the new Usergrid version:
 #
-#   Login to the Tomcat instance and run this command, specifying both 
superuser and tenant organization admin creds:
+# ##
+# STEP 1 - BEFORE SWITCHING TRAFFIC TO NEW UG VERSION
+# ##
 #
-#   python migrate_entity_data.py --org  --super : 
--admin : --init
+# python migrate_entity_data.py --user adminuser:adminpass
 #
-#   This command will setup and bootstrap the database, setup the migration 
system and update index mappings:
-#   - /system/database/setup
-#   - /system/database/bootstrap
-#   - /system/migrate/run/migration-system
-#   - /system/migrate/run/index_mapping_migration
+# The above command performs an appinfo migration and system re-index only.  
This creates indices in Elasticsearch with
+# the updated indexing strategy in the new Usergrid version.
 #
-#   Then it will migrate appinfos, re-index the management app and then for 
each of the specified org's apps
-#   it will de-dup connections and re-index the app.
+# ##
+# STEP 2 - AFTER SWITCHING TRAFFIC TO NEW UG VERSION
+# ##
 #
-#   Write down the 'Re-index start' timestamp when this is finished.
+# python migrate_entity_data.py --user adminuser:adminpass --delta --date 

 #
-# STEP 2 - PUT TENANT ONE TOMCATS IN SERVICE AND DO DELTA MIGRATION
-#
-#   On the same Tomcat instance and run this command with the --date timestamp 
you noted in the previous step:
-#
-#   python migrate_entity_data.py --org  --super : 
--admin : --date 
-#
-#   Then it will migrate appinfos, re-index the management app and then for 
each of the specified org's apps
-#   it will de-dup connections and re-index the app with a start-date 
specified so only data modified since
-#   STEP 1 will be re-indexed.
-#
-# STEP 3 - SETUP TENANT TWO TOMCAT RUNNING 2.1 NOT IN SERVICE
-#
-#   Login to the Tomcat instance and run this command, specifying both 
superuser and tenant organization admin creds:
-#
-#   python migrate_entity_data.py --org  --super : 
--admin :
-#
-#   This command will migrate appinfos, re-index the management app and then 
for each of the specified org's apps
-#   it will de-dup connections and re-index the app.
-#
-#   Write down the 'Re-index start' timestamp when this is finished.
+# The above command performs an appinfo migration, system re-index using a 
start date, and full data migration which
+# includes entity data.  This step is necessary to ensure Usergrid starts 
reading and writing data from the latest
+# entity version, including delta indexing of any documents create during the 
time between STEP 1 and STEP 2.  If
+# all data has already been migrated (running this a 2nd, 3rd, etc. time), 
then the appinfo migration will be skipped.
+
 
-# STEP 4 - PUT TENANT TWO TOMCATS IN SERVICE AND DO DELTA MIGRATION
-#
-#   On the same Tomcat instance and run this command with the --date timestamp 
you noted in the previous step:
-#
-#   python migrate_entity_data.py --org  --super : 
--admin : --date 
-#
-#   Then it will migrate appinfos, re-index the management app and then for 
each of the specified org's apps
-#   it will de-dup connections and re-index the app with a start-date 
specified so only data modified since
-#   STEP 1 will be re-indexed.
-#
-# STEP 5 - FULL 

[14/50] [abbrv] usergrid git commit: Updates the message flow to allow for multiple processor threads per SQS take thread

[toddnine]

Updates the message flow to allow for multiple processor threads per SQS take 
thread


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/76476f17
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/76476f17
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/76476f17

Branch: refs/heads/USERGRID-909
Commit: 76476f17cf8e8be6f01660db3d21110eda8247f5
Parents: 1b43bda
Author: Todd Nine 
Authored: Tue Oct 27 14:35:34 2015 -0600
Committer: Todd Nine 
Committed: Tue Oct 27 14:35:34 2015 -0600

--
 .../asyncevents/AmazonAsyncEventService.java| 68 +++-
 .../index/IndexProcessorFig.java| 13 +++-
 2 files changed, 51 insertions(+), 30 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/76476f17/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java
--
diff --git 
a/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java
 
b/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java
index d93e304..6b9abbc 100644
--- 
a/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java
+++ 
b/stack/core/src/main/java/org/apache/usergrid/corepersistence/asyncevents/AmazonAsyncEventService.java
@@ -679,35 +679,45 @@ public class AmazonAsyncEventService implements 
AsyncEventService {
 }
 while ( true );
 }
-} )
-//this won't block our read loop, just reads and 
proceeds
-.map( messages -> {
-if ( messages == null || messages.size() == 0 
) {
-return null;
-}
-
-try {
-List indexEventResults = 
callEventHandlers( messages );
-List messagesToAck = 
submitToIndex( indexEventResults );
-if ( messagesToAck == null || 
messagesToAck.size() == 0 ) {
-logger.error( "No messages came back 
from the queue operation should have seen "
-+ messages.size(), messages );
-return messagesToAck;
-}
-if ( messagesToAck.size() < 
messages.size() ) {
-logger.error( "Missing messages from 
queue post operation", messages,
-messagesToAck );
-}
-//ack each message, but only if we didn't 
error.
-ack( messagesToAck );
-return messagesToAck;
-}
-catch ( Exception e ) {
-logger.error( "failed to ack messages to 
sqs", e );
-return null;
-//do not rethrow so we can process all of 
them
-}
-} );
+} )//this won't block our read loop, just reads 
and proceeds
+.flatMap( sqsMessages -> {
+
+//do this on a different schedule, and introduce 
concurrency with flatmap for faster processing
+return Observable.just( sqsMessages )
+
+ .map( messages -> {
+ if ( messages == null || 
messages.size() == 0 ) {
+ return null;
+ }
+
+ try {
+ List 
indexEventResults =
+ callEventHandlers( 
messages );
+ List 
messagesToAck =
+ submitToIndex( 
indexEventResults );
+ if ( messagesToAck == 
null || messagesToAck.size() == 0 ) {
+ logger.error(
+ "No messages came 
back 

[25/50] [abbrv] usergrid git commit: Added --super and --admin so we can specify both superuser and org admin credentials. Did some testing and migration seems to work for one tenant, have not tried t

[toddnine]

Added --super and --admin so we can specify both superuser and org admin 
credentials.
Did some testing and migration seems to work for one tenant, have not tried two 
yet.
Commented out the de-dup calls because de-dup never finishes.


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/471dc359
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/471dc359
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/471dc359

Branch: refs/heads/USERGRID-909
Commit: 471dc359d5920ea521c4058e7829a37494a17041
Parents: d79bf4c
Author: Dave Johnson 
Authored: Thu Oct 29 10:07:33 2015 -0400
Committer: Dave Johnson 
Committed: Thu Oct 29 10:07:33 2015 -0400

--
 stack/scripts/migrate_entity_data.py | 130 --
 1 file changed, 87 insertions(+), 43 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/471dc359/stack/scripts/migrate_entity_data.py
--
diff --git a/stack/scripts/migrate_entity_data.py 
b/stack/scripts/migrate_entity_data.py
index f576108..0edd319 100644
--- a/stack/scripts/migrate_entity_data.py
+++ b/stack/scripts/migrate_entity_data.py
@@ -20,7 +20,9 @@
 #
 # STEP 1 - SETUP TENANT ONE TOMCAT RUNNING 2.1 NOT IN SERVICE AND INIT 
MIGRATION
 #
-#   python migrate_entity_data.py --org  --user 
: --init
+#   Login to the Tomcat instance and run this command, specifying both 
superuser and tenant organization admin creds:
+#
+#   python migrate_entity_data.py --org  --super : 
--admin : --init
 #
 #   This command will setup and bootstrap the database, setup the migration 
system and update index mappings:
 #   - /system/database/setup
@@ -31,24 +33,34 @@
 #   Then it will migrate appinfos, re-index the management app and then for 
each of the specified org's apps
 #   it will de-dup connections and re-index the app.
 #
+#   Write down the 'Re-index start' timestamp when this is finished.
+#
 # STEP 2 - PUT TENANT ONE TOMCATS IN SERVICE AND DO DELTA MIGRATION
 #
-#   python migrate_entity_data.py --org  --user 
: --date
+#   On the same Tomcat instance and run this command with the --date timestamp 
you noted in the previous step:
+#
+#   python migrate_entity_data.py --org  --super : 
--admin : --date 
 #
 #   Then it will migrate appinfos, re-index the management app and then for 
each of the specified org's apps
 #   it will de-dup connections and re-index the app with a start-date 
specified so only data modified since
 #   STEP 1 will be re-indexed.
 #
-# STEP 3 - SETUP TENENT TWO TOMCAT RUNNING 2.1 NOT IN SERVICE
+# STEP 3 - SETUP TENANT TWO TOMCAT RUNNING 2.1 NOT IN SERVICE
+#
+#   Login to the Tomcat instance and run this command, specifying both 
superuser and tenant organization admin creds:
 #
-#   python migrate_entity_data.py --org  --user 
: --date
+#   python migrate_entity_data.py --org  --super : 
--admin :
 #
 #   This command will migrate appinfos, re-index the management app and then 
for each of the specified org's apps
 #   it will de-dup connections and re-index the app.
 #
+#   Write down the 'Re-index start' timestamp when this is finished.
+
 # STEP 4 - PUT TENANT TWO TOMCATS IN SERVICE AND DO DELTA MIGRATION
 #
-#   python migrate_entity_data.py --org  --user 
: --date
+#   On the same Tomcat instance and run this command with the --date timestamp 
you noted in the previous step:
+#
+#   python migrate_entity_data.py --org  --super : 
--admin : --date 
 #
 #   Then it will migrate appinfos, re-index the management app and then for 
each of the specified org's apps
 #   it will de-dup connections and re-index the app with a start-date 
specified so only data modified since
@@ -56,7 +68,9 @@
 #
 # STEP 5 - FULL DATA MIGRATION
 #
-#   python migrate_entity_data.py --user : --full
+#   Login to any Tomcat instance in the cluster and run this command (admin 
user creds must be specificed but will be ignored):
+#
+#   python migrate_entity_data.py --super : --admin 
: --full
 #
 #   This command will run the full data migration.
 #
@@ -87,6 +101,8 @@ PLUGIN_ENTITYDATA = 'collections-entity-data'
 PLUGIN_INDEX_MAPPING = 'index_mapping_migration'
 PLUGIN_CORE_DATA = 'core-data'
 
+MANAGEMENT_APP_ID = 'b6768a08-b5d5-11e3-a495-11ddb1de66c8'
+
 
 
 def parse_args():
@@ -97,8 +113,13 @@ def parse_args():
 type=str,
 default='http://localhost:8080')
 
-parser.add_argument('--user',
-help='System Admin Credentials used to authenticate 
with Usergrid  ',
+parser.add_argument('--super',
+help='Superuser username and creds ',
+type=str,
+required=True)
+
+

[07/50] [abbrv] usergrid git commit: USERGRID-1044: make jsonPath extraction optional to keep users from exiting

[toddnine]

USERGRID-1044: make jsonPath extraction optional to keep users from exiting


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/3c60c49a
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/3c60c49a
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/3c60c49a

Branch: refs/heads/USERGRID-909
Commit: 3c60c49a157be63ec626ff88666472f2d4876554
Parents: 1a6de02
Author: Mike Dunker 
Authored: Tue Oct 27 08:52:17 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 08:52:17 2015 -0700

--
 .../org/apache/usergrid/helpers/Extractors.scala  | 18 +++---
 1 file changed, 11 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/3c60c49a/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala 
b/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
index 136a561..b21f9d7 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/helpers/Extractors.scala
@@ -37,7 +37,7 @@ object Extractors {
 jsonPath("$.cursor").transformOption(extract => {
   //it may or may not be present.  If it is, save it, otherwise save it as 
an empty string
   extract.orElse(Some(""))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -47,7 +47,7 @@ object Extractors {
 jsonPath("$.entities[0].uuid").transformOption(extract => {
   //it may or may not be present.  If it is, save it, otherwise save it as 
an empty string
   extract.orElse(Some(""))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -57,7 +57,7 @@ object Extractors {
 jsonPath("$.entities[0].name").transformOption(extract => {
   //it may or may not be present.  If it is, save it, otherwise save it as 
an empty string
   extract.orElse(Some(""))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -67,21 +67,25 @@ object Extractors {
 jsonPath("$.entities[0].modified").ofType[Long].transformOption(extract => 
{
   //it may or may not be present.  If it is, save it, otherwise save it as 
-1
   extract.orElse(Some(-1))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**
* Will extract the audit entities from the get collection response.
*/
   def extractAuditEntities(saveAsName: String) = {
-
jsonPath("$.entities[*]").ofType[Map[String,Any]].findAll.transformOption(extract
 => { extract.orElse(Some(Seq.empty)) }).saveAs(saveAsName)
+
jsonPath("$.entities[*]").ofType[Map[String,Any]].findAll.transformOption(extract
 => {
+  extract.orElse(Some(Seq.empty))
+}).optional.saveAs(saveAsName)
   }
 
   /**
* Will extract the audit entities from the get collection response.
*/
   def extractAuditEntity(saveAsName: String) = {
-
jsonPath("$.entities[0]").ofType[Map[String,Any]].findAll.transformOption(extract
 => { extract.orElse(Some(Seq.empty)) }).saveAs(saveAsName)
+
jsonPath("$.entities[0]").ofType[Map[String,Any]].findAll.transformOption(extract
 => {
+  extract.orElse(Some(Seq.empty))
+}).optional.saveAs(saveAsName)
   }
 
   /**
@@ -100,7 +104,7 @@ object Extractors {
   def maybeExtractEntities(saveAsName: String) = {
 jsonPath("$.entities").ofType[Seq[Any]].transformOption(extract => {
   extract.orElse(Some(Seq()))
-}).saveAs(saveAsName)
+}).optional.saveAs(saveAsName)
   }
 
   /**

[50/50] [abbrv] usergrid git commit: Merge branch 'master' into USERGRID-909

[toddnine]

Merge branch 'master' into USERGRID-909

Change in logic on compaction causes issues with delete as well.  Need to 
resolve


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/817d7ffb
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/817d7ffb
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/817d7ffb

Branch: refs/heads/USERGRID-909
Commit: 817d7ffb4bcfbf43ca591a6d38d75dc0ad79743f
Parents: fae5f2d 568d35d
Author: Todd Nine 
Authored: Mon Nov 2 15:55:00 2015 -0700
Committer: Todd Nine 
Committed: Mon Nov 2 15:55:00 2015 -0700

--
 README-Docs.md  |13 +
 README.md   |10 +-
 content/bootstrap/bootstrap.min.css | 9 +
 content/community/index.html|22 +-
 content/css/bootflat-extensions.css |   356 +
 content/css/bootflat-square.css |69 +
 content/css/bootflat.css|  1560 +
 content/css/font-awesome.min.css|   405 +
 content/css/usergrid-site.css   |  1553 +
 .../installation/ug2-deploy-to-tomcat.txt   |42 +-
 .../docs/_sources/jersey2skeleton/README.txt| 9 +
 .../creating-and-managing-notifications.txt | 2 +
 .../docs/_sources/rest-endpoints/api-docs.txt   |   653 +-
 .../docs/installation/ug2-deploy-to-tomcat.html |37 +-
 content/docs/jersey2skeleton/README.html|   303 +
 .../creating-and-managing-notifications.html| 1 +
 content/docs/rest-endpoints/api-docs.html   |   533 +-
 content/docs/searchindex.js | 2 +-
 content/favicon.ico |   Bin 0 -> 3989 bytes
 content/font/FontAwesome.otf|   Bin 0 -> 61896 bytes
 content/font/fontawesome-webfont-eot.eot|   Bin 0 -> 37405 bytes
 content/font/fontawesome-webfont-svg.svg|   399 +
 content/font/fontawesome-webfont-ttf.ttf|   Bin 0 -> 79076 bytes
 content/font/fontawesome-webfont-woff.woff  |   Bin 0 -> 43572 bytes
 content/img/alberto.jpg |   Bin 0 -> 16137 bytes
 content/img/alex.png|   Bin 0 -> 40842 bytes
 content/img/apache_usergrid_favicon.png |   Bin 0 -> 10735 bytes
 content/img/apache_usergrid_logo_white.png  |   Bin 0 -> 26418 bytes
 .../img/apache_usergrid_logo_white_small.png|   Bin 0 -> 11905 bytes
 content/img/check_flat/default.png  |   Bin 0 -> 25851 bytes
 content/img/dave.jpg|   Bin 0 -> 14005 bytes
 content/img/ed.jpg  |   Bin 0 -> 20460 bytes
 content/img/egg-logo.png|   Bin 0 -> 9938 bytes
 content/img/github.png  |   Bin 0 -> 8936 bytes
 content/img/grey.png|   Bin 0 -> 37896 bytes
 content/img/intellij.png|   Bin 0 -> 9199 bytes
 content/img/jeff.jpg|   Bin 0 -> 13857 bytes
 content/img/michael_r.jpg   |   Bin 0 -> 10244 bytes
 content/img/mike_d.JPG  |   Bin 0 -> 36443 bytes
 content/img/nate.jpg|   Bin 0 -> 4291 bytes
 content/img/rod.jpg |   Bin 0 -> 40313 bytes
 content/img/scott.jpg   |   Bin 0 -> 8555 bytes
 content/img/shawn.jpg   |   Bin 0 -> 69304 bytes
 content/img/stliu.jpg   |   Bin 0 -> 51303 bytes
 content/img/strong.jpg  |   Bin 0 -> 7434 bytes
 content/img/structure101.png|   Bin 0 -> 6475 bytes
 content/img/sungju.jpg  |   Bin 0 -> 11440 bytes
 content/img/tim.jpg |   Bin 0 -> 7611 bytes
 content/img/todd.jpg|   Bin 0 -> 18142 bytes
 content/img/usergrid-logo.pdf   |   398 +
 content/img/usergrid.png|   Bin 0 -> 21994 bytes
 content/img/usergrid_160.png|   Bin 0 -> 2126 bytes
 content/img/usergrid_200.png|   Bin 0 -> 6397 bytes
 content/img/usergrid_300.png|   Bin 0 -> 16330 bytes
 content/img/usergrid_300_transparent.png|   Bin 0 -> 16308 bytes
 content/img/usergrid_400.png|   Bin 0 -> 8746 bytes
 content/img/usergrid_800.png|   Bin 0 -> 14452 bytes
 content/img/usergrid_card.png   |   Bin 0 -> 23295 bytes
 content/img/usergrid_logo.png   |   Bin 0 -> 118086 bytes
 content/img/usergrid_logo_205_50.png|   Bin 0 -> 7058 bytes
 content/img/usergrid_logo_260_50.png|   Bin 0 -> 8682 bytes
 content/img/usergrid_logo_720.png   |   Bin 

[27/50] [abbrv] usergrid git commit: Adds the ability to get credentials info as superuser for user migration

[toddnine]

Adds the ability to get credentials info as superuser for user migration


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/08e97813
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/08e97813
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/08e97813

Branch: refs/heads/USERGRID-909
Commit: 08e978132375751caf90ebcffde7a20d5e84b206
Parents: 5ed8c7c
Author: Todd Nine 
Authored: Wed Oct 28 15:27:36 2015 -0600
Committer: Todd Nine 
Committed: Thu Oct 29 13:55:38 2015 -0600

--
 .../rest/applications/users/UserResource.java   | 45 ++--
 .../usergrid/management/ManagementService.java  |  2 +
 .../cassandra/ManagementServiceImpl.java| 19 +
 3 files changed, 62 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/08e97813/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
--
diff --git 
a/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
 
b/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
index df88cf0..fb10245 100644
--- 
a/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
+++ 
b/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
@@ -42,6 +42,11 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.context.annotation.Scope;
 import org.springframework.stereotype.Component;
+
+import org.apache.amber.oauth2.common.exception.OAuthProblemException;
+import org.apache.amber.oauth2.common.message.OAuthResponse;
+import org.apache.commons.lang.StringUtils;
+
 import org.apache.usergrid.management.ActivationState;
 import org.apache.usergrid.persistence.CredentialsInfo;
 import org.apache.usergrid.persistence.EntityManager;
@@ -52,13 +57,10 @@ import org.apache.usergrid.rest.ApiResponse;
 import org.apache.usergrid.rest.applications.ServiceResource;
 import org.apache.usergrid.rest.exceptions.RedirectionException;
 import org.apache.usergrid.rest.security.annotations.RequireApplicationAccess;
+import org.apache.usergrid.rest.security.annotations.RequireSystemAccess;
 import org.apache.usergrid.security.oauth.AccessInfo;
 import org.apache.usergrid.security.tokens.exceptions.TokenException;
 
-import org.apache.amber.oauth2.common.exception.OAuthProblemException;
-import org.apache.amber.oauth2.common.message.OAuthResponse;
-import org.apache.commons.lang.StringUtils;
-
 import com.sun.jersey.api.json.JSONWithPadding;
 import com.sun.jersey.api.view.Viewable;
 
@@ -165,6 +167,41 @@ public class UserResource extends ServiceResource {
 return new JSONWithPadding( response, callback );
 }
 
+@GET
+@RequireSystemAccess
+@Path("password")
+public JSONWithPadding getUserPassword(@QueryParam("callback") 
@DefaultValue("callback") String callback )
+throws Exception {
+
+logger.info( "UserResource.setUserPassword" );
+
+
+final ApiResponse response = createApiResponse();
+response.setAction( "get user password" );
+
+final UUID applicationId = getApplicationId();
+final UUID targetUserId = getUserUuid();
+
+if ( applicationId == null ) {
+response.setError( "Application not found" );
+return new JSONWithPadding( response, callback );
+}
+
+if ( targetUserId == null ) {
+response.setError( "User not found" );
+return new JSONWithPadding( response, callback );
+}
+
+final CredentialsInfo credentialsInfo = 
management.getAppUserPasswordRaw( applicationId, targetUserId );
+
+
+response.setProperty( "credentials", credentialsInfo );
+
+
+return new JSONWithPadding( response, callback );
+}
+
+
 
 @PUT
 @Path("credentials")

http://git-wip-us.apache.org/repos/asf/usergrid/blob/08e97813/stack/services/src/main/java/org/apache/usergrid/management/ManagementService.java
--
diff --git 
a/stack/services/src/main/java/org/apache/usergrid/management/ManagementService.java
 
b/stack/services/src/main/java/org/apache/usergrid/management/ManagementService.java
index d69de2e..3f02e5a 100644
--- 
a/stack/services/src/main/java/org/apache/usergrid/management/ManagementService.java
+++ 
b/stack/services/src/main/java/org/apache/usergrid/management/ManagementService.java
@@ -288,6 +288,8 @@ public interface ManagementService {
 public void setAppUserPassword( UUID applicationId, UUID userId, String 
oldPassword, String newPassword )
 throws Exception;
 
+

[01/50] [abbrv] usergrid git commit: First pass. Need to test.

[toddnine]

Repository: usergrid
Updated Branches:
  refs/heads/USERGRID-909 fae5f2d83 -> 817d7ffb4


First pass.  Need to test.


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/e41b5f02
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/e41b5f02
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/e41b5f02

Branch: refs/heads/USERGRID-909
Commit: e41b5f02eb12f72b9c19e5ae079b32d528951b47
Parents: 07d2ad3
Author: Todd Nine 
Authored: Mon Oct 26 16:24:37 2015 -0600
Committer: Todd Nine 
Committed: Mon Oct 26 16:24:37 2015 -0600

--
 .../service/StatusServiceImpl.java  |   3 +-
 .../usergrid/rest/ConnectionResource.java   | 198 +++
 .../apache/usergrid/rest/SystemResource.java|   5 +
 3 files changed, 205 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/e41b5f02/stack/core/src/main/java/org/apache/usergrid/corepersistence/service/StatusServiceImpl.java
--
diff --git 
a/stack/core/src/main/java/org/apache/usergrid/corepersistence/service/StatusServiceImpl.java
 
b/stack/core/src/main/java/org/apache/usergrid/corepersistence/service/StatusServiceImpl.java
index 93fe653..282929e 100644
--- 
a/stack/core/src/main/java/org/apache/usergrid/corepersistence/service/StatusServiceImpl.java
+++ 
b/stack/core/src/main/java/org/apache/usergrid/corepersistence/service/StatusServiceImpl.java
@@ -93,8 +93,9 @@ public class StatusServiceImpl implements StatusService {
 final MapManager mapManager = 
mapManagerFactory.createMapManager(new MapScopeImpl(appId, "status"));
 try {
 String statusVal = mapManager.getString(jobString + statusKey);
+//nothing to emit
 if(statusVal==null){
-subscriber.onNext(null);
+subscriber.onCompleted();
 }else {
 final Map data = 
MAPPER.readValue(mapManager.getString(jobString + dataKey), Map.class);
 final Status status = Status.valueOf(statusVal);

http://git-wip-us.apache.org/repos/asf/usergrid/blob/e41b5f02/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java
--
diff --git 
a/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java 
b/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java
new file mode 100644
index 000..b6a38e1
--- /dev/null
+++ b/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java
@@ -0,0 +1,198 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.usergrid.rest;
+
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.context.annotation.Scope;
+import org.springframework.stereotype.Component;
+
+import org.apache.usergrid.corepersistence.service.ConnectionService;
+import org.apache.usergrid.corepersistence.service.ConnectionServiceImpl;
+import org.apache.usergrid.corepersistence.service.StatusService;
+import org.apache.usergrid.corepersistence.util.CpNamingUtils;
+import org.apache.usergrid.persistence.core.scope.ApplicationScope;
+import org.apache.usergrid.persistence.index.utils.UUIDUtils;
+import org.apache.usergrid.persistence.model.util.UUIDGenerator;
+import org.apache.usergrid.rest.security.annotations.RequireSystemAccess;
+
+import com.google.common.base.Preconditions;
+import com.sun.jersey.api.json.JSONWithPadding;
+
+import rx.Observable;

[06/50] [abbrv] usergrid git commit: Moves system endpoints into the system package

[toddnine]

Moves system endpoints into the system package


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/7a4f36ab
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/7a4f36ab
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/7a4f36ab

Branch: refs/heads/USERGRID-909
Commit: 7a4f36ab9c0e504133f80f4d0ad8a70d115ee4f4
Parents: 3b43624
Author: Todd Nine 
Authored: Tue Oct 27 09:46:06 2015 -0600
Committer: Todd Nine 
Committed: Tue Oct 27 09:46:06 2015 -0600

--
 .../usergrid/rest/ApplicationsResource.java | 178 --
 .../usergrid/rest/ConnectionResource.java   | 199 ---
 .../apache/usergrid/rest/DatabaseResource.java  |  98 --
 .../org/apache/usergrid/rest/IndexResource.java | 328 --
 .../apache/usergrid/rest/MigrateResource.java   | 268 ---
 .../apache/usergrid/rest/SystemResource.java| 108 --
 .../rest/system/ApplicationsResource.java   | 178 ++
 .../rest/system/ConnectionResource.java | 202 +++
 .../usergrid/rest/system/DatabaseResource.java  | 100 ++
 .../usergrid/rest/system/IndexResource.java | 331 +++
 .../usergrid/rest/system/MigrateResource.java   | 270 +++
 .../usergrid/rest/system/SystemResource.java| 108 ++
 12 files changed, 1189 insertions(+), 1179 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/7a4f36ab/stack/rest/src/main/java/org/apache/usergrid/rest/ApplicationsResource.java
--
diff --git 
a/stack/rest/src/main/java/org/apache/usergrid/rest/ApplicationsResource.java 
b/stack/rest/src/main/java/org/apache/usergrid/rest/ApplicationsResource.java
deleted file mode 100644
index 55cf0f6..000
--- 
a/stack/rest/src/main/java/org/apache/usergrid/rest/ApplicationsResource.java
+++ /dev/null
@@ -1,178 +0,0 @@
-/*
- *
- *  * Licensed to the Apache Software Foundation (ASF) under one or more
- *  *  contributor license agreements.  The ASF licenses this file to You
- *  * under the Apache License, Version 2.0 (the "License"); you may not
- *  * use this file except in compliance with the License.
- *  * You may obtain a copy of the License at
- *  *
- *  * http://www.apache.org/licenses/LICENSE-2.0
- *  *
- *  * Unless required by applicable law or agreed to in writing, software
- *  * distributed under the License is distributed on an "AS IS" BASIS,
- *  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- *  * See the License for the specific language governing permissions and
- *  * limitations under the License.  For additional information regarding
- *  * copyright in this work, please see the NOTICE file in the top level
- *  * directory of this distribution.
- *
- */
-package org.apache.usergrid.rest;
-
-import com.sun.jersey.api.json.JSONWithPadding;
-import org.apache.usergrid.corepersistence.service.StatusService;
-import org.apache.usergrid.persistence.Entity;
-import org.apache.usergrid.persistence.EntityManager;
-import org.apache.usergrid.persistence.core.util.StringUtils;
-import org.apache.usergrid.persistence.model.util.UUIDGenerator;
-import org.apache.usergrid.rest.security.annotations.RequireSystemAccess;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.context.annotation.Scope;
-import org.springframework.stereotype.Component;
-
-import javax.ws.rs.*;
-import javax.ws.rs.core.Context;
-import javax.ws.rs.core.MediaType;
-import javax.ws.rs.core.UriInfo;
-import java.util.HashMap;
-import java.util.LinkedHashMap;
-import java.util.Map;
-import java.util.UUID;
-import java.util.concurrent.atomic.AtomicInteger;
-
-/**
- * Classy class class.
- */
-@Component
-@Scope( "singleton" )
-@Produces( {
-MediaType.APPLICATION_JSON, "application/javascript", 
"application/x-javascript", "text/ecmascript",
-"application/ecmascript", "text/jscript"
-} )
-public class ApplicationsResource extends AbstractContextResource {
-
-private static final Logger logger = 
LoggerFactory.getLogger(ApplicationsResource.class);
-
-
-public ApplicationsResource() {
-
-logger.info( "ApplicationsResource initialized" );
-} {
-
-}
-
-@RequireSystemAccess
-@DELETE
-@Path( "{applicationId}" )
-public JSONWithPadding clearApplication( @Context UriInfo ui,
- @PathParam("applicationId") UUID 
applicationId,
- @QueryParam( 
"confirmApplicationName" ) String confirmApplicationName,
- @QueryParam( "limit" ) int limit,
- @QueryParam( "callback" ) 
@DefaultValue( 

[48/50] [abbrv] usergrid git commit: USERGRID-1044: update README

[toddnine]

USERGRID-1044: update README


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/73dcaebd
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/73dcaebd
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/73dcaebd

Branch: refs/heads/USERGRID-909
Commit: 73dcaebd286fad3edffd0a8e523b31b042b061c2
Parents: 247f796
Author: Mike Dunker 
Authored: Mon Nov 2 08:54:18 2015 -0800
Committer: Mike Dunker 
Committed: Mon Nov 2 08:54:18 2015 -0800

--
 stack/loadtests/README.md | 12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/73dcaebd/stack/loadtests/README.md
--
diff --git a/stack/loadtests/README.md b/stack/loadtests/README.md
index ec34b42..9dc24e9 100644
--- a/stack/loadtests/README.md
+++ b/stack/loadtests/README.md
@@ -9,6 +9,9 @@ The test scripts are found in the top level loadtests 
directory. Look inside the
 ###testConfig.sh
 Contains defaults that are used for all the other test scripts.
 
+###runAuditDeleteEntities.sh
+For a specified organization and given CSV file, delete all entities, writing 
those that fail to delete to another CSV file.
+
 ###runAuditGetAllAppCollectionEntities.sh
 For a specified organization, finds all apps, and for each app, finds all 
collections and writes a CSV file line containing collection name, UUID, entity 
name, and modified timestamp for each entity in each collection.
 
@@ -16,7 +19,7 @@ For a specified organization, finds all apps, and for each 
app, finds all collec
 For a specified organization and application, finds all collections and writes 
a CSV file line containing collection name, UUID, entity name, and modified 
timestamp for each entity in each collection.
 
 ###runAuditVerifyCollectionEntities.sh
-For a specified organization and given CSV file, verify that all entities are 
retrievable, writing those that fail to another CSV file.
+For a specified organization and given CSV file, verify that all entities are 
retrievable via query, writing those that fail to another CSV file, optionally 
deleting verified entities.
 
 ###runCollectionQueryTest.sh
 For a given collection, retrieve all entities using a cursor and a query.
@@ -115,6 +118,13 @@ Defaults listed are those that are specified by the 
Usergrid Gatling code, not n
 * getViaQuery (**false**) - retrieve entities via query instead of via name or 
uuid
 * queryParams (**""**) - additional query parameters (currently used for get 
by entity or by name)
 * csvFeedPattern (**"random"**) - pattern to use when feeding from a CSV 
("random" is random, "circular" goes through CSV sequentially and restarts from 
beginning when it reaches the end)
+* unlimitedFeed (**false**) - continue loading with no limit on number of 
entities (forces interleavedWorkerFeed=true); hit CTRL-c to abort
+* flushCsv (**0**) - if > 0, flush CSV file when that number of CSV entries 
has been received
+* interleavedWorkerFeed (**false**) - for multiple gatling servers, shard the 
entities via interleaving rather than splitting into separate chunks; for 
example, for 10 workers over 1M entities, interleaving would cause one worker 
to have 1, 11, 21, 31, etc. instead of 1-10 
+* newCsvOnFlush (**false**) - when the output CSV file is flushed, create a 
new CSV file
+* deleteAfterSuccessfulAudit (**false**) - delete a record that is 
successfully verified via query
+* usergridRegion (**""**) - send specified region in the UsergridRegion header 
for all requests -- this allows Edge proxy to call different regions for testing
+* saveInvalidResponse (**false**) - on audit, do not retry but immediately 
save failure to output CSV file
 
 The following settings are currently not used (were used by deprecated tests, 
but may be valid in the future):
 

[02/50] [abbrv] usergrid git commit: Fixes bug in resources and tests job status retrieval successfully

[toddnine]

Fixes bug in resources and tests job status retrieval successfully


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/3b436243
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/3b436243
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/3b436243

Branch: refs/heads/USERGRID-909
Commit: 3b436243eb19ea03151aedf1ea13a63e2048f2f4
Parents: e41b5f0
Author: Todd Nine 
Authored: Mon Oct 26 18:24:58 2015 -0600
Committer: Todd Nine 
Committed: Mon Oct 26 18:24:58 2015 -0600

--
 .../java/org/apache/usergrid/rest/ConnectionResource.java | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/3b436243/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java
--
diff --git 
a/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java 
b/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java
index b6a38e1..dfa4781 100644
--- a/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java
+++ b/stack/rest/src/main/java/org/apache/usergrid/rest/ConnectionResource.java
@@ -43,6 +43,7 @@ import 
org.apache.usergrid.corepersistence.service.ConnectionServiceImpl;
 import org.apache.usergrid.corepersistence.service.StatusService;
 import org.apache.usergrid.corepersistence.util.CpNamingUtils;
 import org.apache.usergrid.persistence.core.scope.ApplicationScope;
+import org.apache.usergrid.persistence.index.query.Identifier;
 import org.apache.usergrid.persistence.index.utils.UUIDUtils;
 import org.apache.usergrid.persistence.model.util.UUIDGenerator;
 import org.apache.usergrid.rest.security.annotations.RequireSystemAccess;
@@ -92,7 +93,7 @@ public class ConnectionResource extends 
AbstractContextResource {
 
 @RequireSystemAccess
 @GET
-@Path( "dedup/{jobId}" )
+@Path( "dedup/{jobId: " + Identifier.UUID_REX + "}" )
 public JSONWithPadding rebuildIndexesGet( @PathParam( "jobId" ) String 
jobId,
   @QueryParam( "callback" ) 
@DefaultValue( "callback" ) String callback )
 throws Exception {
@@ -153,10 +154,10 @@ public class ConnectionResource extends 
AbstractContextResource {
  }};
 
  statusService.setStatus( 
CpNamingUtils.MANAGEMENT_APPLICATION_ID, jobId,
- StatusService.Status.INPROGRESS, status );
+ StatusService.Status.INPROGRESS, status 
).toBlocking().lastOrDefault( null );
  } ).doOnSubscribe( () -> {
 statusService.setStatus( CpNamingUtils.MANAGEMENT_APPLICATION_ID, 
jobId, StatusService.Status.STARTED,
-new HashMap<>() );
+new HashMap<>() ).toBlocking().lastOrDefault( null );
 } ).doOnCompleted( () -> {
 
 final long runningTotal = count.get();

[09/50] [abbrv] usergrid git commit: Merge commit 'refs/pull/414/head' of github.com:apache/usergrid into 2.1-release

[toddnine]

Merge commit 'refs/pull/414/head' of github.com:apache/usergrid into 2.1-release


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/e018c1e0
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/e018c1e0
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/e018c1e0

Branch: refs/heads/USERGRID-909
Commit: e018c1e0d65e3af339e36c69a4e59fe2d2a83167
Parents: 2aec8a0 7a4f36a
Author: Michael Russo 
Authored: Tue Oct 27 10:54:04 2015 -0700
Committer: Michael Russo 
Committed: Tue Oct 27 10:54:04 2015 -0700

--
 .../service/StatusServiceImpl.java  |   3 +-
 .../usergrid/rest/ApplicationsResource.java | 178 --
 .../apache/usergrid/rest/DatabaseResource.java  |  98 --
 .../org/apache/usergrid/rest/IndexResource.java | 328 --
 .../apache/usergrid/rest/MigrateResource.java   | 268 ---
 .../apache/usergrid/rest/SystemResource.java| 103 --
 .../rest/system/ApplicationsResource.java   | 178 ++
 .../rest/system/ConnectionResource.java | 202 +++
 .../usergrid/rest/system/DatabaseResource.java  | 100 ++
 .../usergrid/rest/system/IndexResource.java | 331 +++
 .../usergrid/rest/system/MigrateResource.java   | 270 +++
 .../usergrid/rest/system/SystemResource.java| 108 ++
 12 files changed, 1191 insertions(+), 976 deletions(-)
--

[38/50] [abbrv] usergrid git commit: Addresses comments in review

[toddnine]

Addresses comments in review


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/3d120578
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/3d120578
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/3d120578

Branch: refs/heads/USERGRID-909
Commit: 3d1205784528b5026695910943243a7a38fcbdde
Parents: ba629da
Author: Todd Nine 
Authored: Fri Oct 30 10:51:58 2015 -0600
Committer: Todd Nine 
Committed: Fri Oct 30 10:51:58 2015 -0600

--
 .../apache/usergrid/rest/applications/users/UserResource.java  | 6 +++---
 .../rest/applications/collection/users/UserResourceIT.java | 2 --
 2 files changed, 3 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/3d120578/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
--
diff --git 
a/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
 
b/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
index 1116469..fb2962e 100644
--- 
a/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
+++ 
b/stack/rest/src/main/java/org/apache/usergrid/rest/applications/users/UserResource.java
@@ -173,11 +173,11 @@ public class UserResource extends ServiceResource {
 public JSONWithPadding getUserCredentials(@QueryParam("callback") 
@DefaultValue("callback") String callback )
 throws Exception {
 
-logger.info( "UserResource.setUserPassword" );
+logger.info( "UserResource.getUserCredentials" );
 
 
 final ApiResponse response = createApiResponse();
-response.setAction( "get user password" );
+response.setAction( "get user credentials" );
 
 final UUID applicationId = getApplicationId();
 final UUID targetUserId = getUserUuid();
@@ -210,7 +210,7 @@ public class UserResource extends ServiceResource {
@QueryParam("callback") 
@DefaultValue("callback") String callback )
 throws Exception {
 
-logger.info( "UserResource.setUserPassword" );
+logger.info( "UserResource.setUserCredentials" );
 
 if ( json == null ) {
 return null;

http://git-wip-us.apache.org/repos/asf/usergrid/blob/3d120578/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
--
diff --git 
a/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
 
b/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
index deda5eb..f258f94 100644
--- 
a/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
+++ 
b/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
@@ -1154,7 +1154,6 @@ public class UserResourceIT extends AbstractRestIT {
 //here we modify the hash a little, this way we can break password 
validation, then re-set it to ensure we're actually updating the credentials 
info correctly.
 final String borkedSecret = originalSecret.substring( 0, 
originalSecret.length() -1 );
 
-credentials.put( "credentials", borkedSecret );
 credentials.put( "secret", borkedSecret );
 
 //now PUT it
@@ -1192,7 +1191,6 @@ public class UserResourceIT extends AbstractRestIT {
 //now put the correct one
 
 
-credentials.put( "credentials", originalSecret );
 credentials.put( "secret", originalSecret );
 
 

[32/50] [abbrv] usergrid git commit: Ensure that status is updated properly.

[toddnine]

Ensure that status is updated properly.


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/f8c703c0
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/f8c703c0
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/f8c703c0

Branch: refs/heads/USERGRID-909
Commit: f8c703c02c1182ad63ad86587749eb1ae09c202a
Parents: 471dc35
Author: Dave Johnson 
Authored: Thu Oct 29 17:57:32 2015 -0400
Committer: Dave Johnson 
Committed: Thu Oct 29 17:57:32 2015 -0400

--
 .../rest/system/ConnectionResource.java | 48 +---
 1 file changed, 32 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/f8c703c0/stack/rest/src/main/java/org/apache/usergrid/rest/system/ConnectionResource.java
--
diff --git 
a/stack/rest/src/main/java/org/apache/usergrid/rest/system/ConnectionResource.java
 
b/stack/rest/src/main/java/org/apache/usergrid/rest/system/ConnectionResource.java
index 6e683ed..14b79f3 100644
--- 
a/stack/rest/src/main/java/org/apache/usergrid/rest/system/ConnectionResource.java
+++ 
b/stack/rest/src/main/java/org/apache/usergrid/rest/system/ConnectionResource.java
@@ -55,6 +55,7 @@ import com.google.common.base.Preconditions;
 import com.sun.jersey.api.json.JSONWithPadding;
 
 import rx.Observable;
+import rx.functions.Action1;
 import rx.schedulers.Schedulers;
 
 
@@ -146,36 +147,51 @@ public class ConnectionResource extends 
AbstractContextResource {
 
 //start de duping and run in the background
 connectionService.deDupeConnections( applicationScopeObservable 
).buffer( 10, TimeUnit.SECONDS, 1000 )
- .doOnNext( buffer -> {
+ .doOnNext(buffer -> {
 
 
- final long runningTotal = count.addAndGet( 
buffer.size() );
+ final long runningTotal = 
count.addAndGet(buffer.size());
 
  final Map status = new 
HashMap() {{
- put( "countProcessed", runningTotal );
- put( "updatedTimestamp", 
System.currentTimeMillis() );
+ put("countProcessed", runningTotal);
+ put("updatedTimestamp", 
System.currentTimeMillis());
  }};
 
- statusService.setStatus( 
CpNamingUtils.MANAGEMENT_APPLICATION_ID, jobId,
- StatusService.Status.INPROGRESS, status 
).toBlocking().lastOrDefault( null );
- } ).doOnSubscribe( () -> {
-statusService.setStatus( CpNamingUtils.MANAGEMENT_APPLICATION_ID, 
jobId, StatusService.Status.STARTED,
-new HashMap<>() ).toBlocking().lastOrDefault( null );
-} ).doOnCompleted( () -> {
+ 
statusService.setStatus(CpNamingUtils.MANAGEMENT_APPLICATION_ID, jobId,
+ StatusService.Status.INPROGRESS, 
status).toBlocking().lastOrDefault(null);
+ }).doOnSubscribe(() -> {
+
+statusService.setStatus(CpNamingUtils.MANAGEMENT_APPLICATION_ID,
+jobId, StatusService.Status.STARTED, new 
HashMap<>()).toBlocking().lastOrDefault(null);
+
+}).doOnCompleted(() -> {
 
 final long runningTotal = count.get();
 
 final Map status = new HashMap() {{
-put( "countProcessed", runningTotal );
-put( "updatedTimestamp", System.currentTimeMillis() );
+put("countProcessed", runningTotal);
+put("updatedTimestamp", System.currentTimeMillis());
 }};
 
-statusService
-.setStatus( CpNamingUtils.MANAGEMENT_APPLICATION_ID, jobId, 
StatusService.Status.COMPLETE, status );
-} ).subscribeOn( Schedulers.newThread() ).subscribe();
+statusService.setStatus(CpNamingUtils.MANAGEMENT_APPLICATION_ID,
+jobId, StatusService.Status.COMPLETE, 
status).toBlocking().lastOrDefault(null);
+
+}).doOnError( (throwable) -> {
+logger.error("Error deduping connections", throwable);
+
+final Map status = new HashMap() {{
+put("error", throwable.getMessage() );
+}};
+
+statusService.setStatus(CpNamingUtils.MANAGEMENT_APPLICATION_ID,
+jobId, StatusService.Status.FAILED, 
status).toBlocking().lastOrDefault(null);;
+
+} ).subscribeOn(Schedulers.newThread()).subscribe();
+
 
+final StatusService.JobStatus status =
+   

[18/50] [abbrv] usergrid git commit: USERGRID-1044: for org/app/collection create, default to testConfig settings

[toddnine]

USERGRID-1044: for org/app/collection create, default to testConfig settings


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/fc23b79e
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/fc23b79e
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/fc23b79e

Branch: refs/heads/USERGRID-909
Commit: fc23b79e9a8a08bc57d25e3e60a9718304f8f1c0
Parents: 0389744
Author: Mike Dunker 
Authored: Tue Oct 27 16:39:33 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 16:39:33 2015 -0700

--
 stack/loadtests/runOrgAppSetup.sh | 17 ++---
 1 file changed, 10 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/fc23b79e/stack/loadtests/runOrgAppSetup.sh
--
diff --git a/stack/loadtests/runOrgAppSetup.sh 
b/stack/loadtests/runOrgAppSetup.sh
index 19936b7..b26eb37 100755
--- a/stack/loadtests/runOrgAppSetup.sh
+++ b/stack/loadtests/runOrgAppSetup.sh
@@ -29,6 +29,9 @@ if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 #ENTITY_TYPE=
 #ENTITY_PREFIX=
 #ENTITY_SEED=  #may be overridden on command line
+#ORG= #may be overridden on command line
+#APP= #may be overridden on command line
+#COLLECTION= #may be overridden on command line
 #RETRY_COUNT=
 #ENTITY_PROGRESS_COUNT=
 #CONSTANT_USERS_PER_SEC=
@@ -36,14 +39,14 @@ if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi
 
 die() { echo "$@" 1>&2 ; exit 1; }
 
-[ "$#" -ge 2 ] || die "At least 2 arguments required, $# provided.  Example is 
$0 ORG APP [COLLECTION [SANDBOX_COLLECTION (true/false)]]"
+[ "$#" -ge 1 ] || die "At least 1 argument required, $# provided.  Example is 
$0 SANDBOX_COLLECTION(true/false) [ORG [APP [COLLECTION]]]"
 
-ORG="$1"
-APP="$2"
-COLLECTION="gatlingitems"
-[ "$#" -ge 3 ] && COLLECTION="$3"
-SANDBOX_COLLECTION=true
-[ "$#" -ge 4 ] && SANDBOX_COLLECTION="$4"
+
+SANDBOX_COLLECTION="$1"
+# org, app, and collection can come from testConfig.sh
+[ "$#" -ge 2 ] && ORG="$2"
+[ "$#" -ge 3 ] && APP="$3"
+[ "$#" -ge 4 ] && COLLECTION="$4"
 
 shift $#
 

[30/50] [abbrv] usergrid git commit: This closes #409 -- invalid PR

[toddnine]

This closes #409 -- invalid PR


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/5109952c
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/5109952c
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/5109952c

Branch: refs/heads/USERGRID-909
Commit: 5109952c14dd51a5b3b8c4f6926a8a105d3f6af0
Parents: 690110e
Author: Mike Dunker 
Authored: Thu Oct 29 14:00:32 2015 -0700
Committer: Mike Dunker 
Committed: Thu Oct 29 14:00:32 2015 -0700

--

--

[31/50] [abbrv] usergrid git commit: Adds test to prove functionality of GET + PUT

[toddnine]

Adds test to prove functionality of GET + PUT


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/ba629da5
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/ba629da5
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/ba629da5

Branch: refs/heads/USERGRID-909
Commit: ba629da577235c0c9bedf4addcb44bfb5fdbdccb
Parents: 4784b34
Author: Todd Nine 
Authored: Thu Oct 29 15:25:42 2015 -0600
Committer: Todd Nine 
Committed: Thu Oct 29 15:25:42 2015 -0600

--
 .../collection/users/UserResourceIT.java| 109 +++
 1 file changed, 109 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/ba629da5/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
--
diff --git 
a/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
 
b/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
index 79b8c85..deda5eb 100644
--- 
a/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
+++ 
b/stack/rest/src/test/java/org/apache/usergrid/rest/applications/collection/users/UserResourceIT.java
@@ -37,9 +37,15 @@ import org.apache.usergrid.rest.applications.utils.UserRepo;
 import org.apache.usergrid.utils.UUIDUtils;
 
 import com.sun.jersey.api.client.ClientResponse.Status;
+import com.sun.jersey.api.client.GenericType;
 import com.sun.jersey.api.client.UniformInterfaceException;
+import com.sun.jersey.api.client.WebResource;
+import com.sun.jersey.api.client.filter.HTTPBasicAuthFilter;
+
 import java.io.IOException;
 
+import javax.ws.rs.core.MediaType;
+
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertNotNull;
@@ -1103,4 +1109,107 @@ public class UserResourceIT extends AbstractRestIT {
 
 
assertEquals(response.getResponse().getEntities().get(0).get("uuid").toString(),
 userId.toString());
 }
+
+
+
+@Test
+public void testCredentialsTransfer() throws Exception {
+
+usersResource.post(new User("test_1", "Test1 User", "tes...@test.com", 
"test123")); // client.setApiUrl(apiUrl);
+refreshIndex();
+
+//Entity appInfo = this.app().get().getResponse().getEntities().get(0);
+
+Token token = this.app().token().post(new Token("test_1", "test123"));
+
+assertNotNull(token.getAccessToken());
+
+final String superUserName = this.clientSetup.getSuperuserName();
+final String superUserPassword = 
this.clientSetup.getSuperuserPassword();
+
+
+//get the credentials info
+final CollectionEndpoint collection  = 
userResource.entity("test_1").collection( "credentials" );
+
+final WebResource resource  = collection.getResource();
+
+resource.addFilter( new HTTPBasicAuthFilter(superUserName, 
superUserPassword) );
+
+
+
+final ApiResponse response = resource.type( 
MediaType.APPLICATION_JSON_TYPE)
+ .accept( 
MediaType.APPLICATION_JSON ).get( 
org.apache.usergrid.rest.test.resource.model.ApiResponse.class );
+
+
+//now get the credentials sub object
+
+final Map credentials = ( Map ) 
response.getProperties().get( "credentials" );
+
+
+
+//get out the hash and change it so we can validate
+final String originalSecret = ( String ) credentials.get( "secret" );
+
+
+//here we modify the hash a little, this way we can break password 
validation, then re-set it to ensure we're actually updating the credentials 
info correctly.
+final String borkedSecret = originalSecret.substring( 0, 
originalSecret.length() -1 );
+
+credentials.put( "credentials", borkedSecret );
+credentials.put( "secret", borkedSecret );
+
+//now PUT it
+
+
+final Map> wrapper = new HashMap<>(  );
+wrapper.put( "credentials", credentials );
+
+final WebResource putResource  = collection.getResource();
+
+putResource.addFilter( new HTTPBasicAuthFilter(superUserName, 
superUserPassword) );
+
+
+putResource.type( MediaType.APPLICATION_JSON_TYPE)
+   .accept( MediaType.APPLICATION_JSON ).put(
+org.apache.usergrid.rest.test.resource.model.ApiResponse.class, 
wrapper );
+
+
+//now try to get a password, it should fail because the hash is no 
longer correct
+
+int status = 0;
+
+// bad access token
+try {
+this.app().token().post(new Token("test_1", "test123"));
+

[23/50] [abbrv] usergrid git commit: Fix failing test to ensure we don't set the same timestamp for all entities inserted into the index (since we don't do actually do that). Also ensure it's a long a

[toddnine]

Fix failing test to ensure we don't set the same timestamp for all entities 
inserted into the index (since we don't do actually do that).  Also ensure it's 
a long as what's required of the timestamps


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/c652171f
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/c652171f
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/c652171f

Branch: refs/heads/USERGRID-909
Commit: c652171f476f6ad953c94328e0fb2f4ea0b3aa44
Parents: 22beca2
Author: Michael Russo 
Authored: Wed Oct 28 16:56:19 2015 -0700
Committer: Michael Russo 
Committed: Wed Oct 28 16:56:19 2015 -0700

--
 .../persistence/index/impl/EntityIndexTest.java   | 14 ++
 1 file changed, 6 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/c652171f/stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/impl/EntityIndexTest.java
--
diff --git 
a/stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/impl/EntityIndexTest.java
 
b/stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/impl/EntityIndexTest.java
index 5243d5a..d6758dd 100644
--- 
a/stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/impl/EntityIndexTest.java
+++ 
b/stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/impl/EntityIndexTest.java
@@ -31,6 +31,7 @@ import java.util.concurrent.atomic.AtomicLong;
 import com.google.common.base.Optional;
 import org.apache.usergrid.persistence.core.astyanax.CassandraFig;
 import org.apache.usergrid.persistence.index.*;
+import org.apache.usergrid.persistence.model.field.*;
 import org.junit.Before;
 import org.junit.Rule;
 import org.junit.Test;
@@ -48,10 +49,6 @@ import org.apache.usergrid.persistence.index.utils.UUIDUtils;
 import org.apache.usergrid.persistence.model.entity.Entity;
 import org.apache.usergrid.persistence.model.entity.Id;
 import org.apache.usergrid.persistence.model.entity.SimpleId;
-import org.apache.usergrid.persistence.model.field.ArrayField;
-import org.apache.usergrid.persistence.model.field.IntegerField;
-import org.apache.usergrid.persistence.model.field.StringField;
-import org.apache.usergrid.persistence.model.field.UUIDField;
 import org.apache.usergrid.persistence.model.util.EntityUtils;
 import org.apache.usergrid.persistence.model.util.UUIDGenerator;
 
@@ -394,7 +391,6 @@ public class EntityIndexTest extends BaseIT {
 int numberOfEntities = 1000;
 int versionToSearchFor = numberOfEntities / 2;
 
-IndexEdge searchEdge = new IndexEdgeImpl( appId, "mehCars", 
SearchEdge.NodeType.SOURCE, 1 );
 
 UUID entityUUID = UUID.randomUUID();
 Id entityId = new SimpleId( "mehCar" );
@@ -408,9 +404,11 @@ public class EntityIndexTest extends BaseIT {
 Entity[] entity = new Entity[numberOfEntities];
 for(int i = 0; i < numberOfEntities; i++) {
 entity[i] = EntityIndexMapUtils.fromMap( entityMap );
-EntityUtils.setId( entity[i], entityId );
-EntityUtils.setVersion( entity[i], UUIDGenerator.newTimeUUID() );
-entity[i].setField( new UUIDField( 
IndexingUtils.ENTITY_ID_FIELDNAME, entityUUID ) );
+EntityUtils.setId(entity[i], entityId);
+EntityUtils.setVersion(entity[i], UUIDGenerator.newTimeUUID());
+entity[i].setField(new 
UUIDField(IndexingUtils.ENTITY_ID_FIELDNAME, entityUUID));
+
+IndexEdge searchEdge = new IndexEdgeImpl( appId, "mehCars", 
SearchEdge.NodeType.SOURCE, System.currentTimeMillis()*1000 );
 
 //index the new entity. This is where the loop will be set to 
create like 100 entities.
 indexProducer.put(entityIndex.createBatch().index( searchEdge, 
entity[i]  ).build()).subscribe();

[36/50] [abbrv] usergrid git commit: Add dup edges so we exercise the dedup logic.

[toddnine]

Add dup edges so we exercise the dedup logic.


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/9ada6ea3
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/9ada6ea3
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/9ada6ea3

Branch: refs/heads/USERGRID-909
Commit: 9ada6ea3732a3bfb72cc6a48d12353d0ac832ce6
Parents: b816a18
Author: Dave Johnson 
Authored: Fri Oct 30 08:30:18 2015 -0400
Committer: Dave Johnson 
Committed: Fri Oct 30 08:30:18 2015 -0400

--
 stack/scripts/create_test_data.py | 4 
 1 file changed, 4 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/9ada6ea3/stack/scripts/create_test_data.py
--
diff --git a/stack/scripts/create_test_data.py 
b/stack/scripts/create_test_data.py
index ef07254..3a913ca 100644
--- a/stack/scripts/create_test_data.py
+++ b/stack/scripts/create_test_data.py
@@ -155,6 +155,8 @@ class Creator:
 targetType = appName + "_col1"
 url = appUrl + "/" + sourceCollection + "/" + sourceName + 
"/has/" + targetType + "/" + targetName
 r = requests.post(url=url + "?access_token=" + accessToken)
+r = requests.post(url=url + "?access_token=" + accessToken)
+r = requests.post(url=url + "?access_token=" + accessToken)
 if ( r.status_code >= 400 ):
 print "Error connecting entity " + sourceName + " to " 
+ targetName + ": " + r.text
 print "url is: " + url
@@ -168,6 +170,8 @@ class Creator:
 targetType = appName + "_col2"
 url = appUrl + "/" + sourceCollection + "/" + sourceName + 
"/has/" + targetType + "/" + targetName
 r = requests.post(url=url + "?access_token=" + accessToken)
+r = requests.post(url=url + "?access_token=" + accessToken)
+r = requests.post(url=url + "?access_token=" + accessToken)
 if ( r.status_code >= 400 ):
 print "Error connecting entity " + sourceName + " to " 
+ targetName + ": " + r.text
 print "url is: " + url

[12/50] [abbrv] usergrid git commit: USERGRID-1044: save previous status and error in audit failure CSV

[toddnine]

USERGRID-1044: save previous status and error in audit failure CSV


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/be0033ea
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/be0033ea
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/be0033ea

Branch: refs/heads/USERGRID-909
Commit: be0033eac3f715f63ec716a43ee4d28bffed33dc
Parents: 9158772
Author: Mike Dunker 
Authored: Tue Oct 27 12:56:05 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 12:56:05 2015 -0700

--
 .../usergrid/scenarios/AuditScenarios.scala   | 18 +++---
 .../org/apache/usergrid/settings/Settings.scala   | 12 +++-
 2 files changed, 18 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/be0033ea/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
index 7a93eb6..86c0b73 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
@@ -32,7 +32,7 @@ object AuditScenarios {
 
   //The value for the cursor
   val SessionVarCursor: String = "cursor"
-  val SessionVarStatus: String = "status"
+  val SessionVarStatus: String = "newStatus"
   val SessionVarEntityUuid: String = "entityUuid"
   val SessionVarEntityName: String = "entityName"
   val SessionVarDeletedUuid: String = "deletedUuid"
@@ -157,6 +157,8 @@ object AuditScenarios {
 val modified = session("modified").as[String].toLong
 val uuid = session("uuid").as[String]
 val reqName = session("name").as[String]
+val prevStatus = session("status").as[Int]
+val prevError = session("error").as[String]
 if (status == 200 || status == 404) {
   val collectionEntities = 
session(SessionVarCollectionEntities).as[Seq[Any]]
   val entityUuid = session(SessionVarEntityUuid).as[String]
@@ -164,12 +166,12 @@ object AuditScenarios {
 
   val count = collectionEntities.length
   if (count < 1) {
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"NotFoundAtAll")
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"NotFoundAtAll", prevStatus, prevError)
 Settings.incAuditNotFoundAtAll()
 println(s"NOT FOUND AT ALL: $collectionName.$reqName ($uuid)")
   } else if (count > 1) {
 // invalid
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"QueryInvalidCount$count")
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"QueryInvalidCount$count", prevStatus, prevError)
 Settings.incAuditBadResponse()
 println(s"INVALID RESPONSE (count=$count): 
$collectionName.$reqName ($uuid)")
   } else {
@@ -197,13 +199,13 @@ object AuditScenarios {
   println(s"PAYLOAD NAME MISMATCH (DIRECT): requestedName=$reqName 
returnedName=$entityName")
 }
 
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, errorString)
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, errorString, prevStatus, prevError)
 Settings.incAuditNotFoundViaQuery()
 println(s"NOT FOUND VIA QUERY: $collectionName.$reqName ($uuid)")
   }
   session
 } else if (saveFailures) {
-  Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, "Failure")
+  Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, "Failure", prevStatus, prevError)
   session
 } else {
   session.markAsFailed
@@ -225,6 +227,8 @@ object AuditScenarios {
 val uuid = session("uuid").as[String]
 val reqName = session("name").as[String]
 val modified = session("modified").as[String].toLong
+val prevStatus = session("status").as[Int]
+val prevError = session("error").as[String]
 val collectionName = session(SessionVarCollectionName).as[String]
 val entityUuid = session(SessionVarEntityUuid).as[String]
 val entityName = session(SessionVarEntityName).as[String]
@@ -232,7 +236,7 @@ object AuditScenarios {
 if (count < 1) {
   // will check to see whether accessible directly
 } else if (count > 1) {
-  Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, 

[16/50] [abbrv] usergrid git commit: USERGRID-1044: fixes

[toddnine]

USERGRID-1044: fixes


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/83d7a6f8
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/83d7a6f8
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/83d7a6f8

Branch: refs/heads/USERGRID-909
Commit: 83d7a6f83e005ee8a4794ed86f4f0de95653360c
Parents: be0033e
Author: Mike Dunker 
Authored: Tue Oct 27 13:44:33 2015 -0700
Committer: Mike Dunker 
Committed: Tue Oct 27 13:44:33 2015 -0700

--
 .../usergrid/datagenerators/FeederGenerator.scala |  3 ++-
 .../usergrid/scenarios/AuditScenarios.scala   | 18 --
 .../org/apache/usergrid/settings/Settings.scala   | 10 +-
 3 files changed, 15 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/83d7a6f8/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
index 73b3d3c..38db19a 100755
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/datagenerators/FeederGenerator.scala
@@ -279,11 +279,12 @@ object FeederGenerator {
   val name = if (line != null) array(1) else ""
   val uuid = if (line != null) array(2) else ""
   val modified = if (line != null) array(3) else ""
+  val lastStatus = if (line != null) array(4) else ""
   //println(s"$collectionName|$name|$uuid|$modified")
   val accessField = if (uuid != "") uuid else name
   val queryField = if (uuid != "") s"uuid='$uuid'" else s"name='$name'"
 
-  Map("collectionName" -> collectionName, "name" -> name,  "uuid" -> uuid, 
"modified" -> modified,
+  Map("collectionName" -> collectionName, "name" -> name,  "uuid" -> uuid, 
"modified" -> modified, "lastStatus" -> lastStatus,
 "validEntity" -> validEntity, "accessField" -> accessField, 
"queryField" -> queryField)
 }
   }

http://git-wip-us.apache.org/repos/asf/usergrid/blob/83d7a6f8/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
--
diff --git 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
index 86c0b73..1ce2bc2 100644
--- 
a/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
+++ 
b/stack/loadtests/src/main/scala/org/apache/usergrid/scenarios/AuditScenarios.scala
@@ -157,8 +157,7 @@ object AuditScenarios {
 val modified = session("modified").as[String].toLong
 val uuid = session("uuid").as[String]
 val reqName = session("name").as[String]
-val prevStatus = session("status").as[Int]
-val prevError = session("error").as[String]
+val lastStatus = session("lastStatus").as[String]
 if (status == 200 || status == 404) {
   val collectionEntities = 
session(SessionVarCollectionEntities).as[Seq[Any]]
   val entityUuid = session(SessionVarEntityUuid).as[String]
@@ -166,12 +165,12 @@ object AuditScenarios {
 
   val count = collectionEntities.length
   if (count < 1) {
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"NotFoundAtAll", prevStatus, prevError)
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"NotFoundAtAll", lastStatus)
 Settings.incAuditNotFoundAtAll()
 println(s"NOT FOUND AT ALL: $collectionName.$reqName ($uuid)")
   } else if (count > 1) {
 // invalid
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"QueryInvalidCount$count", prevStatus, prevError)
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, s"QueryInvalidCount$count", lastStatus)
 Settings.incAuditBadResponse()
 println(s"INVALID RESPONSE (count=$count): 
$collectionName.$reqName ($uuid)")
   } else {
@@ -199,13 +198,13 @@ object AuditScenarios {
   println(s"PAYLOAD NAME MISMATCH (DIRECT): requestedName=$reqName 
returnedName=$entityName")
 }
 
-Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, errorString, prevStatus, prevError)
+Settings.addAuditUuid(uuid, collectionName, reqName, modified, 
status, errorString, lastStatus)
 Settings.incAuditNotFoundViaQuery()
  

[39/50] [abbrv] usergrid git commit: Add call to database bootstrap

[toddnine]

Add call to database bootstrap


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/9b589d37
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/9b589d37
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/9b589d37

Branch: refs/heads/USERGRID-909
Commit: 9b589d37bba835cdc8fc8912919f7929fa1bfb21
Parents: 8737f69
Author: Dave Johnson 
Authored: Fri Oct 30 13:49:25 2015 -0400
Committer: Dave Johnson 
Committed: Fri Oct 30 13:49:25 2015 -0400

--
 stack/scripts/multitenant_migrate.py | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/usergrid/blob/9b589d37/stack/scripts/multitenant_migrate.py
--
diff --git a/stack/scripts/multitenant_migrate.py 
b/stack/scripts/multitenant_migrate.py
index 6955de7..5f1918e 100644
--- a/stack/scripts/multitenant_migrate.py
+++ b/stack/scripts/multitenant_migrate.py
@@ -226,6 +226,7 @@ class Migrate:
 # Init the migration system as this is the first migration 
done on the cluster
 
 self.run_database_setup()
+self.run_database_bootstrap()
 
 migration_system_updated = self.is_migration_system_updated()
 

[15/50] [abbrv] usergrid git commit: Merge branch 'delete-event-updates' into 2.1-release

[toddnine]

Merge branch 'delete-event-updates' into 2.1-release


Project: http://git-wip-us.apache.org/repos/asf/usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/usergrid/commit/26860545
Tree: http://git-wip-us.apache.org/repos/asf/usergrid/tree/26860545
Diff: http://git-wip-us.apache.org/repos/asf/usergrid/diff/26860545

Branch: refs/heads/USERGRID-909
Commit: 26860545ece19361569370bc63496a01bb48738d
Parents: e018c1e 76476f1
Author: Todd Nine 
Authored: Tue Oct 27 14:40:23 2015 -0600
Committer: Todd Nine 
Committed: Tue Oct 27 14:40:23 2015 -0600

--
 .../asyncevents/AmazonAsyncEventService.java| 68 ++
 .../asyncevents/AsyncEventService.java  |  5 ++
 .../asyncevents/InMemoryAsyncEventService.java  |  5 ++
 .../index/IndexProcessorFig.java| 13 +++-
 .../read/traverse/AbstractReadGraphFilter.java  | 72 +---
 .../traverse/ReadGraphCollectionFilter.java | 10 ++-
 .../traverse/ReadGraphConnectionFilter.java | 10 ++-
 .../impl/stage/NodeDeleteListenerImpl.java  | 27 +++-
 8 files changed, 150 insertions(+), 60 deletions(-)
--