Hi Doug, Thanks for the information. I'll rebuild the package removing the contrib section. -- Regards,
Sreejith K On Wed, Jun 23, 2010 at 9:56 PM, Doug Judd <[email protected]> wrote: > Hi Sreejith, > > It looks like these issues have to do with building the "contrib" section. > The main thing in the contrib section is Mateusz's original MapReduce > connector. However, the new Thrift-based MapReduce connectors > in src/java/MapReduce/org/hypertable/hadoop supersede it. Try dropping the > contrib/ section from the toplevel CMakeLists.txt file. > > One thing to verify after you build and install Hypertable, but prior to > creating the package, is that the Hypertable .jar files appear in > /opt/hypertable/<version>/lib/java. If not, that would be the source of the > problem. The Hypertable .jar files get built from source, so be sure that > they build properly. You can run "make java" to just build the .jar files. > > - Doug > > On Wed, Jun 23, 2010 at 6:02 AM, Sreejith K <[email protected]> wrote: > >> Hi Doug, >> >> I built those debian packages (for Stanislav) from 0.9.3.2 source. Here is >> what I followed. >> >> I made changes to hypertable sources according to issues (as it wasn't >> building without the fix) >> http://code.google.com/p/hypertable/issues/detail?id=220 >> http://code.google.com/p/hypertable/issues/detail?id=409 >> >> s...@codedesk:~$ cmake -DPACKAGE_THRIFTBROKER=1 -DBUILD_SHARED_LIBS=ON >> -DCMAKE_BUILD_TYPE=Release ~/src/hypertable >> -DJAVA_INCLUDE_PATH=/usr/lib/jvm/java-6-sun-1.6.0.20/include >> -DJAVA_INCLUDE_PATH2=/usr/lib/jvm/java-6-sun-1.6.0.20/include/linux >> -DHADOOP_INCLUDE_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/include >> -DHADOOP_LIB_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/lib >> -DPHPTHRIFT_ROOT=/home/sree/src/thrift-0.2.0/lib/php/src >> s...@codedesk:~$ cpack -G DEB >> s...@codedesk:~$ cpack -G RPM >> >> s...@codedesk:~$ cmake -DPACKAGE_THRIFTBROKER= -DBUILD_SHARED_LIBS=ON >> -DCMAKE_BUILD_TYPE=Release ~/src/hypertable >> -DJAVA_INCLUDE_PATH=/usr/lib/jvm/java-6-sun-1.6.0.20/include >> -DJAVA_INCLUDE_PATH2=/usr/lib/jvm/java-6-sun-1.6.0.20/include/linux >> -DHADOOP_INCLUDE_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/include >> -DHADOOP_LIB_PATH=/home/sree/src/hadoop-0.20.2/src/c++/install/lib >> -DPHPTHRIFT_ROOT=/home/sree/src/thrift-0.2.0/lib/php/src >> s...@codedesk:~$ cpack -G DEB >> s...@codedesk:~$ cpack -G RPM >> >> s...@codedesk:~$ sudo dpkg -i hypertable-0.9.3.2-linux-i386.deb >> s...@codedesk:~$ ls -l /opt/hypertable/0.9.3.2/lib/ >> total 16560 >> drwxr-xr-x 2 sree sree 4096 2010-06-23 18:01 java >> -rw-r--r-- 1 sree sree 75640 2010-06-15 19:38 >> libboost_filesystem.so.1.40.0 >> -rw-r--r-- 1 sree sree 42696 2010-06-15 19:46 >> libboost_iostreams.so.1.40.0 >> -rw-r--r-- 1 sree sree 289336 2010-06-15 19:46 >> libboost_program_options.so.1.40.0 >> -rw-r--r-- 1 sree sree 13800 2010-06-15 19:46 libboost_system.so.1.40.0 >> -rw-r--r-- 1 sree sree 83764 2010-06-15 19:46 libboost_thread.so.1.40.0 >> -rw-r--r-- 1 sree sree 70076 2010-06-15 19:31 libbz2.so.1.0 >> -rw-r--r-- 1 sree sree 1763912 2010-06-15 19:31 libdb_cxx-4.8.so >> -rw-r--r-- 1 sree sree 91768 2010-06-15 19:31 libevent-1.4.so.2 >> -rw-r--r-- 1 sree sree 157064 2010-06-15 19:31 libexpat.so.1 >> -rw-r--r-- 1 sree sree 116272 2010-06-15 19:31 libgcc_s.so.1 >> -rw-r--r-- 1 sree sree 527962 2010-06-15 19:46 libHyperCommon.so >> -rw-r--r-- 1 sree sree 442913 2010-06-15 19:48 libHyperComm.so >> -rw-r--r-- 1 sree sree 348999 2010-06-15 19:51 libHyperDfsBroker.so >> -rw-r--r-- 1 sree sree 53546 2010-06-15 20:00 libHyperDfsCmds.so >> -rw-r--r-- 1 sree sree 1789883 2010-06-15 19:59 libHyperRanger.so >> -rw-r--r-- 1 sree sree 665860 2010-06-15 19:48 libHyperspace.so >> -rw-r--r-- 1 sree sree 2779668 2010-06-15 19:53 libHypertable.so >> -rw-r--r-- 1 sree sree 93367 2010-06-15 20:02 libHyperThriftConfig.so >> -rw-r--r-- 1 sree sree 780921 2010-06-15 20:02 libHyperThrift.so >> -rw-r--r-- 1 sree sree 95871 2010-06-15 19:48 libHyperTools.so >> -rw-r--r-- 1 sree sree 245256 2010-06-15 19:31 liblog4cpp.so.5 >> -rw-r--r-- 1 sree sree 212010 2010-06-15 20:03 libMapReduce.so >> -rw-r--r-- 1 sree sree 223768 2010-06-15 19:31 libncurses.so.5 >> -rw-r--r-- 1 sree sree 204860 2010-06-15 19:31 libreadline.so.6 >> -rw-r--r-- 1 sree sree 206952 2010-06-15 19:31 librrd.so.4 >> -rw-r--r-- 1 sree sree 232969 2010-06-15 19:31 libsigar-x86-linux.so >> -rw-r--r-- 1 sree sree 962800 2010-06-15 19:31 libstdc++.so.6 >> -rw-r--r-- 1 sree sree 681611 2010-06-15 19:31 libtcmalloc_minimal.so.0 >> -rw-r--r-- 1 sree sree 308151 2010-06-15 19:31 libthriftnb.so.0 >> -rw-r--r-- 1 sree sree 3112270 2010-06-15 19:31 libthrift.so.0 >> -rw-r--r-- 1 sree sree 79512 2010-06-15 19:31 libz.so.1 >> drwxr-xr-x 4 sree sree 4096 2010-06-23 18:01 perl >> drwxr-xr-x 3 sree sree 4096 2010-06-23 18:01 php >> drwxr-xr-x 4 sree sree 4096 2010-06-23 18:01 py >> drwxr-xr-x 3 sree sree 4096 2010-06-23 18:01 rb >> s...@codedesk:~$ ls -l /opt/hypertable/0.9.3.2/lib/java >> total 5880 >> -rw-r--r-- 1 sree sree 41123 2010-06-15 16:50 commons-cli-1.2.jar >> -rw-r--r-- 1 sree sree 38015 2010-06-15 16:50 commons-logging-1.0.4.jar >> -rw-r--r-- 1 sree sree 2689741 2010-06-15 16:50 hadoop-0.20.2-core.jar >> -rw-r--r-- 1 sree sree 11358 2010-06-15 16:50 >> hadoop-0.20.2-core.LICENSE.txt >> -rw-r--r-- 1 sree sree 1567530 2010-06-15 16:50 hbase-0.20.4.jar >> -rw-r--r-- 1 sree sree 106547 2010-06-15 16:50 junit-4.3.1.jar >> -rw-r--r-- 1 sree sree 15162 2010-06-15 16:50 junit-4.3.1.LICENSE.txt >> -rw-r--r-- 1 sree sree 195759 2010-06-15 16:50 libthrift-0.3.0.jar >> -rw-r--r-- 1 sree sree 358180 2010-06-15 16:50 log4j-1.2.13.jar >> -rw-r--r-- 1 sree sree 23445 2010-06-15 16:50 slf4j-api-1.5.8.jar >> -rw-r--r-- 1 sree sree 9679 2010-06-15 16:50 slf4j-log4j12-1.5.8.jar >> -rw-r--r-- 1 sree sree 915651 2010-06-15 16:50 zookeeper-3.2.2.jar >> >> -- >> Regards, >> >> Sreejith K >> >> >> >> On Wed, Jun 23, 2010 at 2:33 PM, Stanislav Yudin <[email protected]>wrote: >> >>> It's ubuntu 10.04, so I missed that initially. >>> sta...@unicorn:/opt/hypertable/0.9.3.2$ uname -a >>> Linux unicorn 2.6.32-21-server #32-Ubuntu SMP Fri Apr 16 09:17:34 UTC >>> 2010 x86_64 GNU/Linux >>> sta...@unicorn:/opt/hypertable/0.9.3.2$ lsb_release -a >>> No LSB modules are available. >>> Distributor ID: Ubuntu >>> Description: Ubuntu 10.04 LTS >>> Release: 10.04 >>> Codename: lucid >>> sta...@unicorn:/opt/hypertable/0.9.3.2$ >>> >>> >>> On Wed, Jun 23, 2010 at 2:27 PM, Doug Judd <[email protected]> wrote: >>> >>>> That change went out a while ago. We could have done a better job >>>> announcing the changes necessary. Replace Hyperspace.Master.* with >>>> Hyperspace.Replica.* We'll fix the document tomorrow. Can you post the >>>> output of uname -a ? >>>> >>>> - Doug >>>> >>>> >>>> On Wed, Jun 23, 2010 at 1:52 AM, Stanislav Yudin <[email protected]>wrote: >>>> >>>>> Ok, putting jars back fixes the problem with dfs broker startup, so >>>>> that mean it was the only reason. Also, I've noticed that you've made >>>>> changes to configuration parsing, namely dropped Hyperspace.Master.* >>>>> properties. Config props >>>>> reference<http://code.google.com/p/hypertable/wiki/ConfigPropetiesReference?tm=1>doesn't >>>>> know anything about new props, so I guess that you've missed >>>>> documentation update. Had I missed the announce of such changes? >>>>> >>>>> >>>>> On Wed, Jun 23, 2010 at 2:14 PM, Stanislav Yudin <[email protected]>wrote: >>>>> >>>>>> Hi Doug, >>>>>> Sorry, but we are back to 0.9.2.7 for our development environment for >>>>>> now. We will try to reproduce problem with missed jars on 0.9.3.3 build >>>>>> on >>>>>> cluster. Thou on my personal test host it did reproduce on 0.9.3.2. See >>>>>> console session. >>>>>> sta...@unicorn:~$ sudo rm -fr /opt/hypertable/* >>>>>> sta...@unicorn:~$ ls -l >>>>>> -rw-r--r-- 1 stanly stanly 10658452 2010-06-19 13:58 >>>>>> hypertable-0.9.2.7-linux-x86_64.deb >>>>>> -rw-r--r-- 1 stanly stanly 13695996 2010-06-18 17:42 >>>>>> hypertable-0.9.3.2-linux-x86_64.deb >>>>>> -rw-r--r-- 1 stanly stanly 3714334 2010-06-19 13:57 >>>>>> hypertable-thriftbroker-0.9.2.7-linux-x86_64.deb >>>>>> -rw-r--r-- 1 stanly stanly 9547270 2010-06-18 17:42 >>>>>> hypertable-thriftbroker-0.9.3.2-linux-x86_64.deb >>>>>> >>>>>> sta...@unicorn:~$ ls /opt/hypertable/ >>>>>> sta...@unicorn:~$ sudo dpkg -i hypertable-0.9.3.2-linux-x86_64.deb >>>>>> Selecting previously deselected package hypertable. >>>>>> (Reading database ... 174153 files and directories currently >>>>>> installed.) >>>>>> >>>>>> Unpacking hypertable (from hypertable-0.9.3.2-linux-x86_64.deb) ... >>>>>> Setting up hypertable (0.9.3.2) ... >>>>>> sta...@unicorn:~$ ls /opt/hypertable/0.9.3.2/lib/java/ >>>>>> commons-cli-1.2.jar hadoop-0.20.2-core.jar >>>>>> hbase-0.20.4.jar junit-4.3.1.LICENSE.txt log4j-1.2.13.jar >>>>>> slf4j-log4j12-1.5.8.jar >>>>>> commons-logging-1.0.4.jar hadoop-0.20.2-core.LICENSE.txt >>>>>> junit-4.3.1.jar libthrift-0.3.0.jar slf4j-api-1.5.8.jar >>>>>> zookeeper-3.2.2.jar >>>>>> sta...@unicorn:~$ dpkg -L hypertable | grep "lib/java" >>>>>> >>>>>> /opt/hypertable/0.9.3.2/lib/java >>>>>> /opt/hypertable/0.9.3.2/lib/java/zookeeper-3.2.2.jar >>>>>> >>>>>> /opt/hypertable/0.9.3.2/lib/java/commons-cli-1.2.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/hbase-0.20.4.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/hadoop-0.20.2-core.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/junit-4.3.1.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/hadoop-0.20.2-core.LICENSE.txt >>>>>> /opt/hypertable/0.9.3.2/lib/java/slf4j-api-1.5.8.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/slf4j-log4j12-1.5.8.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/log4j-1.2.13.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/junit-4.3.1.LICENSE.txt >>>>>> /opt/hypertable/0.9.3.2/lib/java/libthrift-0.3.0.jar >>>>>> /opt/hypertable/0.9.3.2/lib/java/commons-logging-1.0.4.jar >>>>>> sta...@unicorn:~$ >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jun 22, 2010 at 11:52 AM, Doug Judd <[email protected]>wrote: >>>>>> >>>>>>> Hi Stanislav, >>>>>>> >>>>>>> I just installed the Debian package on an 64-bit Ubuntu 8.04 and the >>>>>>> hypertable jar files were deposited in the proper location. I've >>>>>>> attached >>>>>>> the terminal session below. How exactly did you install this package? >>>>>>> Can >>>>>>> you remove the package and try installing it again to be sure that it >>>>>>> is in >>>>>>> fact a problem with the package? Thanks. >>>>>>> >>>>>>> - Doug >>>>>>> >>>>>>> [r...@ip-10-196-57-187 ~]# dpkg -i >>>>>>> hypertable-0.9.3.2-linux-x86_64.deb >>>>>>> Selecting previously deselected package hypertable. >>>>>>> (Reading database ... 32779 files and directories currently >>>>>>> installed.) >>>>>>> Unpacking hypertable (from hypertable-0.9.3.2-linux-x86_64.deb) ... >>>>>>> Setting up hypertable (0.9.3.2) ... >>>>>>> [r...@ip-10-196-57-187 ~]# ls -l /opt/hypertable/0.9.3.2/lib/java/ >>>>>>> total 31020 >>>>>>> -rw-r--r-- 1 root root 41123 2010-06-13 17:55 commons-cli-1.2.jar >>>>>>> -rw-r--r-- 1 root root 38015 2010-06-13 17:55 >>>>>>> commons-logging-1.0.4.jar >>>>>>> -rw-r--r-- 1 root root 2689741 2010-06-13 17:55 >>>>>>> hadoop-0.20.2-core.jar >>>>>>> -rw-r--r-- 1 root root 11358 2010-06-13 17:55 >>>>>>> hadoop-0.20.2-core.LICENSE.txt >>>>>>> -rw-r--r-- 1 root root 1567530 2010-06-13 17:55 hbase-0.20.4.jar >>>>>>> -rw-r--r-- 1 root root 12942187 2010-06-13 18:07 >>>>>>> hypertable-0.9.3.2-examples.jar >>>>>>> -rw-r--r-- 1 root root 12756293 2010-06-13 18:07 >>>>>>> hypertable-0.9.3.2.jar >>>>>>> -rw-r--r-- 1 root root 106547 2010-06-13 17:55 junit-4.3.1.jar >>>>>>> -rw-r--r-- 1 root root 15162 2010-06-13 17:55 >>>>>>> junit-4.3.1.LICENSE.txt >>>>>>> -rw-r--r-- 1 root root 195759 2010-06-13 17:55 libthrift-0.3.0.jar >>>>>>> -rw-r--r-- 1 root root 358180 2010-06-13 17:55 log4j-1.2.13.jar >>>>>>> -rw-r--r-- 1 root root 23445 2010-06-13 17:55 slf4j-api-1.5.8.jar >>>>>>> -rw-r--r-- 1 root root 9679 2010-06-13 17:55 >>>>>>> slf4j-log4j12-1.5.8.jar >>>>>>> -rw-r--r-- 1 root root 915651 2010-06-13 17:55 zookeeper-3.2.2.jar >>>>>>> >>>>>>> >>>>>>> On Sat, Jun 19, 2010 at 1:26 AM, Stanislav Yudin >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> Hi everyone, >>>>>>>> Recently we've upgraded one of the installations to 0.9.3.2 from >>>>>>>> 0.9.2.7, and we've got a problem running it on a hadoop cluster. >>>>>>>> Basically dfs broker cannot load required java classes which are >>>>>>>> supposed to be there from deb. Maybe package doesn't have something. >>>>>>>> Hadoop version 0.20.1 runs fine and contains /hypertable with >>>>>>>> correct rights. >>>>>>>> >>>>>>>> sta...@unicorn:~$ apt-cache show hypertable >>>>>>>> Package: hypertable >>>>>>>> Status: install ok installed >>>>>>>> Priority: optional >>>>>>>> Section: devel >>>>>>>> Installed-Size: 31565 >>>>>>>> Maintainer: [email protected] >>>>>>>> Architecture: amd64 >>>>>>>> Version: 0.9.3.2 >>>>>>>> Description: Hypertable 0.9.3.2 >>>>>>>> >>>>>>>> sta...@unicorn:~$ /opt/hypertable/0.9.3.2/bin/start-dfsbroker.shhadoop >>>>>>>> DFS broker: available file descriptors: 1024 >>>>>>>> Waiting for DFS Broker (hadoop) to come up... >>>>>>>> ... >>>>>>>> sta...@unicorn:~$ cat /opt/hypertable/ >>>>>>>> 0.9.3.2/log/DfsBroker.hadoop.log >>>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>>>>> org/hypertable/DfsBroker/hadoop/main >>>>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>>> org.hypertable.DfsBroker.hadoop.main >>>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >>>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >>>>>>>> Could not find the main class: org.hypertable.DfsBroker.hadoop.main. >>>>>>>> Program will exit. >>>>>>>> sta...@unicorn:~$ ls /opt/hypertable/0.9.3.2/lib/ >>>>>>>> java libboost_thread.so.1.40.0 >>>>>>>> libgcc_s.so.1 libHyperRanger.so libHyperTools.so >>>>>>>> librrd.so.4 libthrift.so.0 rb >>>>>>>> libboost_filesystem.so.1.40.0 libbz2.so.1.0 >>>>>>>> libHyperCommon.so libHyperspace.so liblog4cpp.so.5 >>>>>>>> libsigar-amd64-linux.so libz.so.1 >>>>>>>> libboost_iostreams.so.1.40.0 libdb_cxx-4.8.so >>>>>>>> libHyperComm.so libHypertable.so libMapReduce.so >>>>>>>> libstdc++.so.6 perl >>>>>>>> libboost_program_options.so.1.40.0 libevent-1.4.so.2 >>>>>>>> libHyperDfsBroker.so libHyperThriftConfig.so libncurses.so.5 >>>>>>>> libtcmalloc_minimal.so.0 php >>>>>>>> libboost_system.so.1.40.0 libexpat.so.1 >>>>>>>> libHyperDfsCmds.so libHyperThrift.so libreadline.so.6 >>>>>>>> libthriftnb.so.0 py >>>>>>>> sta...@unicorn:~$ ls /opt/hypertable/0.9.3.2/lib/java/ >>>>>>>> commons-cli-1.2.jar hadoop-0.20.2-core.jar >>>>>>>> hbase-0.20.4.jar junit-4.3.1.LICENSE.txt log4j-1.2.13.jar >>>>>>>> slf4j-log4j12-1.5.8.jar >>>>>>>> commons-logging-1.0.4.jar hadoop-0.20.2-core.LICENSE.txt >>>>>>>> junit-4.3.1. >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Hypertable Development" group. >>>>>>>> To post to this group, send email to >>>>>>>> [email protected]. >>>>>>>> To unsubscribe from this group, send email to >>>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>>> . >>>>>>>> For more options, visit this group at >>>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Hypertable Development" group. >>>>>>> To post to this group, send email to [email protected] >>>>>>> . >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]<hypertable-dev%[email protected]> >>>>>>> . >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>>> >>>>>> >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Hypertable Development" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]<hypertable-dev%[email protected]> >>>>> . >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Hypertable Development" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]<hypertable-dev%[email protected]> >>>> . >>>> For more options, visit this group at >>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Hypertable Development" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<hypertable-dev%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/hypertable-dev?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Hypertable Development" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<hypertable-dev%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/hypertable-dev?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Hypertable Development" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<hypertable-dev%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/hypertable-dev?hl=en. > -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
