Those are from the tracer. In the accumulo-site.xml, change the trace.token.property.password value to your root password. Or you can create a new user in Accumulo and update that and trace.user field appropriately.
On Thu, Sep 12, 2013 at 3:43 PM, John Vines <[email protected]> wrote: > Those are from the tracer. In the accumulo-site.xml, change > the trace.token.property.password value to your root password. Or you can > create a new user in Accumulo and update that and trace.user field > appropriately. > > > On Thu, Sep 12, 2013 at 3:01 PM, Pete Carlson <[email protected]> wrote: > >> Thanks Eric, that worked. I now see in the monitor that I have 1 Tablet >> Server. >> >> Under the monitor log i.e., http://127.0.0.1:50095/log the warn and >> fatal messages went away, and now I have the following error: >> >> tserver:localhost.localdomain 115 ERROR >> ThriftSecurityException(user:root, code:BAD_CREDENTIALS) >> >> and I also see these same errors in the tserver_localhost.localdomain.log >> (here's a snippet): >> >> 2013-09-12 14:56:29,985 [client.ClientServiceHandler] ERROR: >> ThriftSecurityException(user:root, code:BAD_CREDENTIALS) >> 2013-09-12 14:56:30,996 [client.ClientServiceHandler] ERROR: >> ThriftSecurityException(user:root, code:BAD_CREDENTIALS) >> 2013-09-12 14:56:32,008 [client.ClientServiceHandler] ERROR: >> ThriftSecurityException(user:root, code:BAD_CREDENTIALS) >> 2013-09-12 14:56:33,021 [client.ClientServiceHandler] ERROR: >> ThriftSecurityException(user:root, code:BAD_CREDENTIALS) >> >> Do I have to do a similar configuration for Thrift that I just did for >> Hadoop? >> >> Pete >> >> On Thu, Sep 12, 2013 at 2:31 PM, Eric Newton <[email protected]>wrote: >> >>> Add: >>> >>> <property> >>> <name>dfs.support.append</name> >>> <value>true</value> >>> </property> >>> <property> >>> <name>dfs.data.synconclose</name> >>> <value>true</value> >>> </property> >>> >>> To hdfs-site.xml in your hadoop configuration. >>> >>> -Eric >>> >>> >>> >>> On Thu, Sep 12, 2013 at 2:27 PM, Pete Carlson <[email protected]>wrote: >>> >>>> Ok, so now that I have an Accumulo monitor I discovered that my >>>> Accumulo instance doesn't have any tablet servers. >>>> >>>> Here is what I tried so far to resolve the issue: >>>> >>>> 1) Looked in the tserver_localhost.localdomain.log file, and found >>>> this FATAL message: >>>> >>>> 2013-09-12 08:09:42,273 [tabletserver.TabletServer] FATAL: Must set >>>> dfs.durable.sync OR dfs.support.append to true. Which one needs to be set >>>> depends on your version of HDFS. See ACCUMULO-623. >>>> HADOOP RELEASE VERSION SYNC NAME DEFAULT >>>> Apache Hadoop 0.20.205 dfs.support.append false >>>> Apache Hadoop 0.23.x dfs.support.append true >>>> Apache Hadoop 1.0.x dfs.support.append false >>>> Apache Hadoop 1.1.x dfs.durable.sync true >>>> Apache Hadoop 2.0.0-2.0.2 dfs.support.append true >>>> Cloudera CDH 3u0-3u3 ???? true >>>> Cloudera CDH 3u4 dfs.support.append true >>>> Hortonworks HDP `1.0 dfs.support.append false >>>> Hortonworks HDP `1.1 dfs.support.append false >>>> 2013-09-12 11:54:00,752 [server.Accumulo] INFO : tserver starting >>>> 2013-09-12 11:54:00,768 [server.Accumulo] INFO : Instance >>>> d57cdc38-8ceb-4192-9da3-1ce2664df33b >>>> 2013-09-12 11:54:00,771 [server.Accumulo] INFO : Data Version 5 >>>> 2013-09-12 11:54:00,771 [server.Accumulo] INFO : Attempting to talk to >>>> zookeeper >>>> 2013-09-12 11:54:00,952 [server.Accumulo] INFO : Zookeeper connected >>>> and initialized, attemping to talk to HDFS >>>> 2013-09-12 11:54:00,956 [server.Accumulo] INFO : Connected to HDFS >>>> 2013-09-12 11:54:00,969 [server.Accumulo] INFO : gc.cycle.delay = 5m >>>> 2013-09-12 11:54:00,969 [server.Accumulo] INFO : gc.cycle.start = 30s >>>> 2013-09-12 11:54:00,969 [server.Accumulo] INFO : gc.port.client = 50091 >>>> 2013-09-12 11:54:00,969 [server.Accumulo] INFO : gc.threads.delete = 16 >>>> 2013-09-12 11:54:00,969 [server.Accumulo] INFO : gc.trash.ignore = false >>>> >>>> I saw this same FATAL message 8 times in the >>>> tserver_localhost.localdomain.log >>>> between blocks of INFO messages, but no other fatal or warn messages. >>>> Btw, this FATAL message also appears in my >>>> tserver_localhost.localdomain.debug.log file. >>>> >>>> When I googled this Fatal message I found this page: >>>> >>>> http://mail-archives.apache.org/mod_mbox/accumulo-user/201304.mbox/%[email protected]%3E >>>> with >>>> the same "WARN: There are no tablet servers: check that zookeeper and >>>> accumulo are running." message. >>>> >>>> I checked http://127.0.0.1:50095/tservers, and it showed that there >>>> were no tablet servers online. I looked at http://127.0.0.1:50095/log, >>>> and saw the following messages: >>>> >>>> FATAL: Must set dfs.durable.sync or dfs.support.append to true. Which >>>> one needs to be set depends on your version of HDFS. See Accumulo-623. >>>> >>>> WARN: There are no tablet servers: check that zookeeper and accumulo >>>> are running. >>>> >>>> Using the info from the page I referenced above, I checked my >>>> $ACCUMULO_HOME path and realized that I hadn't set that in the >>>> conf/accumulo-env.sh >>>> >>>> So, I set it to the following: >>>> >>>> test -z "$ACCUMULO_HOME" && export >>>> ACCUMULO_HOME=/home/accumulo/accumulo-1.5.0 >>>> >>>> When I did an echo of $ACCUMULO_HOME it didn't return anything, so I >>>> also tried setting it in my bash profile to see if that made any difference >>>> (it didn't). >>>> >>>> I also looked in the lib directory but didn't see any stray jars. >>>> >>>> In my tracer_localhost_localdomain.log I saw the following Exception >>>> with Zookeeper: >>>> >>>> 2013-09-11 16:09:48,649 [impl.ServerClient] WARN : There are no tablet >>>> servers: check that zookeeper and accumulo are running. >>>> 2013-09-11 18:02:23,385 [zookeeper.ZooCache] WARN : Zookeeper error, >>>> will retry >>>> org.apache.zookeeper.KeeperException$SessionExpiredException: >>>> KeeperErrorCode = Session expired for >>>> /accumulo/d57cdc38-8ceb-4192-9da3-1ce2664df33b/tservers >>>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) >>>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >>>> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468) >>>> at org.apache.accumulo.fate.zookeeper.ZooCache$1.run(ZooCache.java:167) >>>> at org.apache.accumulo.fate.zookeeper.ZooCache.retry(ZooCache.java:130) >>>> at >>>> org.apache.accumulo.fate.zookeeper.ZooCache.getChildren(ZooCache.java:178) >>>> at >>>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:140) >>>> at >>>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:128) >>>> at >>>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:123) >>>> at >>>> org.apache.accumulo.core.client.impl.ServerClient.executeRaw(ServerClient.java:105) >>>> at >>>> org.apache.accumulo.core.client.impl.ServerClient.execute(ServerClient.java:71) >>>> at >>>> org.apache.accumulo.core.client.impl.ConnectorImpl.<init>(ConnectorImpl.java:64) >>>> at >>>> org.apache.accumulo.server.client.HdfsZooInstance.getConnector(HdfsZooInstance.java:154) >>>> at >>>> org.apache.accumulo.server.client.HdfsZooInstance.getConnector(HdfsZooInstance.java:149) >>>> at >>>> org.apache.accumulo.server.trace.TraceServer.<init>(TraceServer.java:185) >>>> at >>>> org.apache.accumulo.server.trace.TraceServer.main(TraceServer.java:260) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>> at org.apache.accumulo.start.Main$1.run(Main.java:101) >>>> at java.lang.Thread.run(Thread.java:724) >>>> 2013-09-12 08:09:44,861 [server.Accumulo] INFO : tracer starting >>>> 2013-09-12 08:09:44,926 [server.Accumulo] INFO : Instance >>>> d57cdc38-8ceb-4192-9da3-1ce2664df33b >>>> 2013-09-12 08:09:44,929 [server.Accumulo] INFO : Data Version 5 >>>> 2013-09-12 08:09:44,929 [server.Accumulo] INFO : Attempting to talk to >>>> zookeeper >>>> 2013-09-12 08:09:45,114 [server.Accumulo] INFO : Zookeeper connected >>>> and initialized, attemping to talk to HDFS >>>> 2013-09-12 08:09:45,130 [server.Accumulo] INFO : Connected to HDFS >>>> 2013-09-12 08:09:45,150 [server.Accumulo] INFO : gc.cycle.delay = 5m >>>> 2013-09-12 08:09:45,150 [server.Accumulo] INFO : gc.cycle.start = 30s >>>> >>>> but then it appeared to reconnect with Zookeeper. >>>> >>>> 2) I looked at the ACCUMULO-623 Jira ticket from the FATAL message >>>> above i.e., https://issues.apache.org/jira/browse/ACCUMULO-623 , but >>>> this Jira ticket indicates this issue is fixed in Accumulo 1.5.0 although >>>> that ticket references Hadoop 1.0.3, and Zookeeper 3.3.3 (I'm using Hadoop >>>> 1.2.1, and Zookeeper 3.4.5) I noticed that a fix was added to Hadoop 1.1 >>>> for a related Hadoop Jira ticket. >>>> >>>> 3) Next, I went to the Accumulo Jira page i.e., >>>> https://issues.apache.org/jira/browse/accumulo to look for this >>>> issue. Besides ACCUMULO-623, the following tickets are similar but not >>>> quite the same: >>>> >>>> - ACCUMULO-327 ( but I don't have any tablet servers to begin with >>>> to be killed) >>>> - ACCUMULO-1235 (I only have a the default !METADATA table) >>>> >>>> 4) Looked again at the User manual to see if there was information >>>> about configuring the tablet server, but didn't see anything. >>>> >>>> Any suggestions on what I should try next? >>>> >>>> Thanks, >>>> >>>> Pete >>>> >>> >>> >> > > > -- > Cheers > ~John >
