Hi Max,
Thanks for reaching out.
The intent is to start up an arango cluster. I am having a lot of trouble
with it so I am trying to debug by starting just 1 of each service (agency,
primary, coordinator) on a single machine. Each of these services will get
its own directory e.g. /agency, /primary, /coordinator.
Here's how I am starting up the agency which seems to start up just fine
/usr/sbin/arangod --agency.activate true --server.endpoint tcp://0.0.0.0:25001
--server.authentication false --agency.size 1 --agency.supervision true
--database.directory /data/data/arango/agency
2017-01-23T12:35:29Z [10207] INFO ArangoDB 3.1.8 [linux] 64bit, using VPack
0.1.30, ICU 54.1, V8 5.0.71.39, OpenSSL 1.0.1e-fips 11 Feb 2013
2017-01-23T12:35:29Z [10207] INFO using SSL options:
SSL_OP_CIPHER_SERVER_PREFERENCE, SSL_OP_TLS_ROLLBACK_BUG
2017-01-23T12:35:29Z [10207] INFO file-descriptors (nofiles) hard limit is
999999, soft limit is 999999
2017-01-23T12:35:29Z [10207] INFO created database directory
'/data/data/arango/agency'.
2017-01-23T12:35:29Z [10207] INFO WAL directory
'/data/data/arango/agency/journals' does not exist. creating it...
2017-01-23T12:35:29Z [10207] INFO JavaScript using startup
'//usr/share/arangodb3/js', application '/var/lib/arangodb3-apps'
2017-01-23T12:35:33Z [10207] INFO In database '_system': No version
information file found in database directory.
2017-01-23T12:35:33Z [10207] INFO In database '_system': Found 14 defined
task(s), 12 task(s) to run
2017-01-23T12:35:33Z [10207] INFO In database '_system': state standalone/
init, tasks setupGraphs, setupUsers, createUsersIndex, addDefaultUserSystem,
createModules, createRouting, insertRedirectionsAll, setupAqlFunctions,
createStatistics, createFrontend, setupQueues, setupJobs
2017-01-23T12:35:33Z [10207] INFO In database '_system': init successfully
finished
2017-01-23T12:35:33Z [10207] INFO using endpoint 'http+tcp://0.0.0.0:25001'
for non-encrypted requests
2017-01-23T12:35:38Z [10207] INFO ArangoDB (version 3.1.8 [linux]) is ready
for business. Have fun!
Second I start up the primary db service. this also seems to start up just
fine:
/usr/sbin/arangod --server.authentication=false --server.endpoint
tcp://0.0.0.0:8529
--cluster.my-address tcp://$HOSTNAME:8529 --cluster.my-local-info
$HOSTNAME-primary --cluster.my-role PRIMARY --cluster.agency-endpoint
tcp://$HOSTNAME:25001 --database.directory /data/data/arango/primary
2017-01-23T12:38:49Z [10677] INFO ArangoDB 3.1.8 [linux] 64bit, using VPack
0.1.30, ICU 54.1, V8 5.0.71.39, OpenSSL 1.0.1e-fips 11 Feb 2013
2017-01-23T12:38:49Z [10677] INFO using SSL options:
SSL_OP_CIPHER_SERVER_PREFERENCE, SSL_OP_TLS_ROLLBACK_BUG
2017-01-23T12:38:49Z [10677] INFO file-descriptors (nofiles) hard limit is
999999, soft limit is 999999
2017-01-23T12:38:49Z [10677] INFO created database directory
'/data/data/arango/primary'.
2017-01-23T12:38:49Z [10677] INFO WAL directory
'/data/data/arango/primary/journals' does not exist. creating it...
2017-01-23T12:38:49Z [10677] INFO JavaScript using startup
'//usr/share/arangodb3/js', application '/var/lib/arangodb3-apps'
2017-01-23T12:38:53Z [10677] INFO Cluster feature is turned on. Agency
version: {"server":"arango","version":"3.1.8","license":"community"}, Agency
endpoints: http+tcp://osdi-03:25001, server id: 'DBServer001', internal
address: tcp://osdi-03:8529, role: PRIMARY
2017-01-23T12:38:53Z [10677] INFO using heartbeat interval value '1000 ms'
from agency
2017-01-23T12:38:53Z [10677] INFO In database '_system': No version
information file found in database directory.
2017-01-23T12:38:53Z [10677] INFO In database '_system': Database is
up-to-date (30108/cluster-local/init)
2017-01-23T12:38:53Z [10677] INFO using endpoint 'http+tcp://0.0.0.0:8529'
for non-encrypted requests
2017-01-23T12:38:53Z [10677] INFO bootstraped DB server DBServer001
2017-01-23T12:38:53Z [10677] INFO bootstraped DB server DBServer001
2017-01-23T12:38:53Z [10677] INFO bootstraped DB server DBServer001
2017-01-23T12:38:53Z [10677] INFO bootstraped DB server DBServer001
2017-01-23T12:38:53Z [10677] INFO bootstraped DB server DBServer001
2017-01-23T12:38:53Z [10677] INFO In database '_system': Database is
up-to-date (-/db-server-local/init)
2017-01-23T12:38:53Z [10677] INFO bootstraped DB server DBServer001
2017-01-23T12:38:53Z [10677] INFO bootstraped DB server DBServer001
2017-01-23T12:38:53Z [10677] INFO ArangoDB (version 3.1.8 [linux]) is ready
for business. Have fun!
Lastly starting up the coordinator will result in an error:
/usr/sbin/arangod --server.authentication=false --server.endpoint
tcp://$HOSTNAME:8530
--cluster.my-address tcp://$HOSTNAME:8530 --cluster.my-local-info
$HOSTNAME-coordinator --cluster.my-role COORDINATOR
--cluster.agency-endpoint tcp://osdi-03:25001 /data/data/arango/cordinator
2017-01-23T12:41:28Z [10895] INFO ArangoDB 3.1.8 [linux] 64bit, using VPack
0.1.30, ICU 54.1, V8 5.0.71.39, OpenSSL 1.0.1e-fips 11 Feb 2013
2017-01-23T12:41:28Z [10895] INFO using SSL options:
SSL_OP_CIPHER_SERVER_PREFERENCE, SSL_OP_TLS_ROLLBACK_BUG
2017-01-23T12:41:28Z [10895] INFO Waiting for DBservers to show up...
2017-01-23T12:41:40Z [10895] INFO Waiting for DBservers to show up...
2017-01-23T12:41:41Z [10895] INFO Waiting for DBservers to show up...
2017-01-23T12:41:42Z [10895] INFO Waiting for DBservers to show up...
2017-01-23T12:41:43Z [10895] INFO Waiting for DBservers to show up...
2017-01-23T12:41:43Z [10895] INFO Found 1 DBservers.
2017-01-23T12:41:43Z [10895] INFO file-descriptors (nofiles) hard limit is
999999, soft limit is 999999
2017-01-23T12:41:43Z [10895] INFO created database directory
'/data/data/arango/cordinator'.
2017-01-23T12:41:43Z [10895] INFO WAL directory
'/data/data/arango/cordinator/journals' does not exist. creating it...
2017-01-23T12:41:43Z [10895] INFO JavaScript using startup
'//usr/share/arangodb3/js', application '/var/lib/arangodb3-apps'
2017-01-23T12:41:47Z [10895] INFO Cluster feature is turned on. Agency
version: {"server":"arango","version":"3.1.8","license":"community"}, Agency
endpoints: http+tcp://osdi-03:25001, server id: 'Coordinator001', internal
address: tcp://osdi-03:8530, role: COORDINATOR
2017-01-23T12:41:47Z [10895] INFO using heartbeat interval value '1000 ms'
from agency
2017-01-23T12:41:47Z [10895] INFO In database '_system': No version
information file found in database directory.
2017-01-23T12:41:47Z [10895] INFO In database '_system': Database is
up-to-date (30108/cluster-local/init)
2017-01-23T12:41:47Z [10895] INFO using endpoint 'http+tcp://osdi-03:8530'
for non-encrypted requests
2017-01-23T12:41:47Z [10895] INFO In database '_system': Found 14 defined
task(s), 12 task(s) to run
2017-01-23T12:41:47Z [10895] INFO In database '_system': state
coordinator-global/init, tasks setupGraphs, setupUsers, createUsersIndex,
addDefaultUserSystem, createModules, createRouting, insertRedirectionsAll,
setupAqlFunctions, createStatistics, createFrontend, setupQueues, setupJobs
2017-01-23T12:41:47Z [10895] WARNING {cluster} createCollectionCoordinator:
replicationFactor is too large for the number of DBservers
2017-01-23T12:41:52Z [10895] WARNING {cluster} createCollectionCoordinator:
replicationFactor is too large for the number of DBservers
2017-01-23T12:42:02Z [10895] ERROR In database '_system': Executing task #4
(addDefaultUserSystem: add default root user for system database) failed
with exception: ArangoError 502: could not determine number of documents in
collection (while optimizing plan) ArangoError: could not determine number
of documents in collection (while optimizing plan)
2017-01-23T12:42:02Z [10895] ERROR at Error (native)
2017-01-23T12:42:02Z [10895] ERROR at ArangoStatement.execute
(/usr/share/arangodb3/js/server/modules/@arangodb/arango-statement.js:81:16)
2017-01-23T12:42:02Z [10895] ERROR at ArangoDatabase._query
(/usr/share/arangodb3/js/server/modules/@arangodb/arango-database.js:79:45)
2017-01-23T12:42:02Z [10895] ERROR at SimpleQueryByExample.execute
(/usr/share/arangodb3/js/server/modules/@arangodb/simple-query.js:137:42)
2017-01-23T12:42:02Z [10895] ERROR at
SimpleQueryByExample.SimpleQuery.toArray
(/usr/share/arangodb3/js/common/modules/@arangodb/simple-query-common.js:340:8)
2017-01-23T12:42:02Z [10895] ERROR at ArangoCollection.firstExample
(/usr/share/arangodb3/js/server/modules/@arangodb/arango-collection.js:287:71)
2017-01-23T12:42:02Z [10895] ERROR at Object.exports.save
(/usr/share/arangodb3/js/server/modules/@arangodb/users.js:135:22)
2017-01-23T12:42:02Z [10895] ERROR at Object.addTask.task
(/usr/share/arangodb3/js/server/upgrade-database.js:510:21)
2017-01-23T12:42:02Z [10895] ERROR at runTasks
(/usr/share/arangodb3/js/server/upgrade-database.js:274:27)
2017-01-23T12:42:02Z [10895] ERROR at upgradeDatabase
(/usr/share/arangodb3/js/server/upgrade-database.js:346:16)
2017-01-23T12:42:02Z [10895] ERROR In database '_system': Executing task #4
(addDefaultUserSystem: add default root user for system database) failed.
Aborting init procedure.
2017-01-23T12:42:02Z [10895] ERROR In database '_system': Please fix the
problem and try starting the server again.
2017-01-23T12:42:02Z [10895] ERROR upgrade-database.js for cluster script
failed!
2017-01-23T12:42:02Z [10895] WARNING {cluster} createCollectionCoordinator:
replicationFactor is too large for the number of DBservers
2017-01-23T12:42:12Z [10895] ERROR ArangoError: could not determine number
of documents in collection (while optimizing plan)
2017-01-23T12:42:12Z [10895] ERROR at Error (native)
2017-01-23T12:42:12Z [10895] ERROR at ArangoStatement.execute
(/usr/share/arangodb3/js/server/modules/@arangodb/arango-statement.js:81:16)
2017-01-23T12:42:12Z [10895] ERROR at ArangoDatabase._query
(/usr/share/arangodb3/js/server/modules/@arangodb/arango-database.js:79:45)
2017-01-23T12:42:12Z [10895] ERROR at SimpleQueryAll.execute
(/usr/share/arangodb3/js/server/modules/@arangodb/simple-query.js:96:42)
2017-01-23T12:42:12Z [10895] ERROR at
SimpleQueryAll.SimpleQuery.hasNext
(/usr/share/arangodb3/js/common/modules/@arangodb/simple-query-common.js:388:8)
2017-01-23T12:42:12Z [10895] ERROR at refillCaches
(/usr/share/arangodb3/js/server/modules/@arangodb/foxx/manager.js:265:17)
2017-01-23T12:42:12Z [10895] ERROR at Object.initializeFoxx
(/usr/share/arangodb3/js/server/modules/@arangodb/foxx/manager.js:1483:3)
2017-01-23T12:42:12Z [10895] ERROR at Object.foxxes
(/usr/share/arangodb3/js/server/bootstrap/foxxes.js:64:47)
2017-01-23T12:42:12Z [10895] ERROR at
server/bootstrap/cluster-bootstrap.js:51:54
2017-01-23T12:42:12Z [10895] ERROR at
server/bootstrap/cluster-bootstrap.js:55:2
2017-01-23T12:42:12Z [10895] ERROR JavaScript exception in file
'/usr/share/arangodb3/js/server/modules/@arangodb/foxx/queues/index.js' at
108,7: TypeError: Cannot read property 'save' of undefined
2017-01-23T12:42:12Z [10895] ERROR ! throw err;
2017-01-23T12:42:12Z [10895] ERROR ! ^
2017-01-23T12:42:12Z [10895] FATAL error during execution of JavaScript
file 'server/bootstrap/coordinator.js'
If i try to start the service again I get a more condense error message
/usr/sbin/arangod --server.authentication=false --server.endpoint
tcp://$HOSTNAME:8530
--cluster.my-address tcp://$HOSTNAME:8530 --cluster.my-local-info
$HOSTNAME-coordinator --cluster.my-role COORDINATOR
--cluster.agency-endpoint tcp://osdi-03:25001 /data/data/arango/cordinator
2017-01-23T12:43:35Z [11171] INFO Waiting for DBservers to show up...
2017-01-23T12:43:36Z [11171] INFO Waiting for DBservers to show up...
2017-01-23T12:43:37Z [11171] INFO Waiting for DBservers to show up...
2017-01-23T12:43:38Z [11171] INFO Waiting for DBservers to show up...
2017-01-23T12:43:38Z [11171] INFO Found 1 DBservers.
2017-01-23T12:43:38Z [11171] INFO file-descriptors (nofiles) hard limit is
999999, soft limit is 999999
2017-01-23T12:43:38Z [11171] INFO JavaScript using startup
'//usr/share/arangodb3/js', application '/var/lib/arangodb3-apps'
2017-01-23T12:43:42Z [11171] INFO Cluster feature is turned on. Agency
version: {"server":"arango","version":"3.1.8","license":"community"}, Agency
endpoints: http+tcp://osdi-03:25001, server id: 'Coordinator001', internal
address: tcp://osdi-03:8530, role: COORDINATOR
2017-01-23T12:43:42Z [11171] INFO using heartbeat interval value '1000 ms'
from agency
2017-01-23T12:43:42Z [11171] INFO In database '_system': Database is
up-to-date (30108/cluster-local/existing)
2017-01-23T12:43:42Z [11171] INFO using endpoint 'http+tcp://osdi-03:8530'
for non-encrypted requests
2017-01-23T12:43:42Z [11171] ERROR JavaScript exception in file
'/usr/share/arangodb3/js/server/modules/@arangodb/foxx/queues/index.js' at
108,7: TypeError: Cannot read property 'save' of undefined
2017-01-23T12:43:42Z [11171] ERROR ! throw err;
2017-01-23T12:43:42Z [11171] ERROR ! ^
2017-01-23T12:43:42Z [11171] FATAL error during execution of JavaScript
file 'server/bootstrap/coordinator.js'
I've been struggling / blocked by this for a while now. If you have any
ideas / suggestions it is greatly appreciated.
Thanks,
On Monday, January 23, 2017 at 3:27:50 AM UTC-5, [email protected] wrote:
>
> Hi,
>
> how do you start up your cluster?
>
> The line
>
> > 2017-01-21T22:59:20Z [2378] INFO WAL directory
> '/data/data/arango/cordinator/journals' does not exist. creating it...
>
> indicates that the coordinator does not find its own data directory. Is
> the "cordinator" maybe a typo?
> Furthermore, it seems that your coordinator (maybe as a consequence) is
> only seeing one dbserver (it says "found 1 DBserver"). A consequence is
> that it cannot create the system collections with replicationFactor 2. This
> should be OK, but then something else is happening: "could not determine
> number of documents...". This indicates that your coordinator cannot
> properly talk to the right DBservers.
>
> Has this way to start up a cluster worked before? Is this a restart? I
> fear we need some more details to find out what is happening here.
>
> Cheers,
> Max.
>
--
You received this message because you are subscribed to the Google Groups
"ArangoDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
