[
https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224485#comment-13224485
]
Ferdy Galema commented on GORA-76:
----------------------------------
Ok I got a bit more working now, but not yet fully done.
I solved the jackson issue by excluding it from hadoop (it include jackson
1.0.1) and including it explicitely with specific version 1.4.2 because this
what avro 1.3.3 requires. So that is out of the way.
Next problem is this: When I run the gora-core tests the data directory cannot
be initialized it seems. This is partial output of
org.apache.gora.avro.mapreduce.TestDataFileAvroStoreMapReduce when it goes
right:
Starting DataNode 0 with dfs.data.dir:
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1,/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2
12/03/07 17:26:40 INFO common.Storage: Storage directory
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1 is not
formatted.
12/03/07 17:26:40 INFO common.Storage: Formatting ...
12/03/07 17:26:40 INFO common.Storage: Storage directory
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2 is not
formatted.
12/03/07 17:26:40 INFO common.Storage: Formatting ...
12/03/07 17:26:40 INFO datanode.DataNode: Registered FSDatasetStatusMBean
12/03/07 17:26:40 INFO datanode.DataNode: Opened info server at 35222
12/03/07 17:26:40 INFO datanode.DataNode: Balancing bandwith is 1048576 bytes/s
Now output when it goes wrong (when I apply patch)
Starting DataNode 0 with dfs.data.dir:
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1,/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2
12/03/07 17:33:21 WARN impl.MetricsSystemImpl: Metrics system not started:
Cannot locate configuration: tried hadoop-metrics2-datanode.properties,
hadoop-metrics2.properties
12/03/07 17:33:21 WARN util.MBeans:
Hadoop:service=DataNode,name=MetricsSystem,sub=Control
javax.management.InstanceAlreadyExistsException: MXBean already registered with
name Hadoop:service=NameNode,name=MetricsSystem,sub=Control
at
com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:120)
at
com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:143)
at
com.sun.jmx.mbeanserver.MBeanSupport.preRegister2(MBeanSupport.java:183)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:941)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:56)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.initSystemMBean(MetricsSystemImpl.java:500)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:140)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:40)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1491)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1467)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:417)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
at
org.apache.hadoop.mapred.HadoopTestCase.setUp(HadoopTestCase.java:148)
at
org.apache.gora.mapreduce.DataStoreMapReduceTestBase.setUp(DataStoreMapReduceTestBase.java:54)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
12/03/07 17:33:21 WARN datanode.DataNode: Invalid directory in dfs.data.dir:
Incorrect permission for
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1, expected:
rwxr-xr-x, while actual: rwxrwxr-x
12/03/07 17:33:21 WARN datanode.DataNode: Invalid directory in dfs.data.dir:
Incorrect permission for
/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2, expected:
rwxr-xr-x, while actual: rwxrwxr-x
12/03/07 17:33:21 ERROR datanode.DataNode: All directories in dfs.data.dir are
invalid.
Not sure what exactly is the cause. Note that HBASE-4709 describes a similar
problem. Any help would be useful.
> Upgrade to Hadoop 1.0.0
> -----------------------
>
> Key: GORA-76
> URL: https://issues.apache.org/jira/browse/GORA-76
> Project: Apache Gora
> Issue Type: Improvement
> Components: build process, storage
> Affects Versions: 0.2
> Reporter: Lewis John McGibbney
> Fix For: 0.2
>
> Attachments: GORA-76.patch
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
