Very cool, everyone! I'm looking forward to getting this merged. We've always wanted to get this working. :)
--------- Matt Taylor OS Community Flag-Bearer Numenta On Fri, Aug 23, 2013 at 12:13 PM, Matt Keith <[email protected]> wrote: > Complete success! After dusting off my command line gdb skills, I was > able to trace the error to the apr library and noticed that I had > originally just linked the apr header files to the linux64 include dir. I > didn't realize that the headers had platform specific settings, so I > updated all of the files in linux32arm/include/apr-1. I was even able to > get the changes into the current PR, so hopefully it will all go through. > > After building with the new header files, the testeverything app returns > this: > ******************************************************************* > * Numenta Unit Test summary > * > * Total categories = 40 > * Total tests = 2240 > * Total passed = 2240 > * Total disabled = 0 > * Total failed = 0 > * Total critical failures = 0 > * Success rate = 100 > ******************************************************************* > > Yay! Now I can move on to learning how to actually use the software. :) > > Thank you everyone for your help and encouragement. > > Matt > > On Aug 22, 2013, at 9:50 AM, Matt Keith <[email protected]> wrote: > > Mark, I have submitted PR#202 for all of my code changes and additions. > > Subutai, that is what I thought. Thank you for the clarification. > > When I run htmtest through gdb I get the following output: > (gdb) handle SIGILL nostop > Signal Stop Print Pass to program Description > SIGILL No Yes Yes Illegal instruction > (gdb) r > Starting program: /home/pi/nta/eng/bin/htmtest > <snip> > Adding a PyNode region... > > Program received signal SIGSEGV, Segmentation fault. > nta::Link::initialize (this=0x1, destinationOffset=1600704) at > /home/pi/nupic/nta/engine/Link.cpp:139 > 139 if(dest_->isRegionLevel()) > (gdb) where > #0 nta::Link::initialize (this=0x1, destinationOffset=1600704) at > /home/pi/nupic/nta/engine/Link.cpp:139 > #1 0x00187c20 in ?? () > #2 0x00187c20 in ?? () > Backtrace stopped: previous frame identical to this frame (corrupt stack?) > (gdb) list > 134 else > 135 { > 136 NTA_CHECK(srcD.isDontcare() || srcD == > src_->getRegion().getDimensions()); > 137 } > 138 > 139 if(dest_->isRegionLevel()) > 140 { > 141 Dimensions d; > 142 for(size_t i = 0; i < dest_->getRegion().getDimensions().size(); > i++) > 143 { > (gdb) > > Any ideas why the isRegionLevel() would cause a SEGV? > > Thanks, > > Matt > > On Aug 22, 2013, at 4:35 AM, Marek Otahal <[email protected]> wrote: > > Great work Matt! Thanks. > Can you make a PR and get this stuff upstream once you think the branch is > ready? > thank you, Mark > > > On Thu, Aug 22, 2013 at 6:08 AM, Subutai Ahmad <[email protected]>wrote: > >> >> That's excellent! >> >> You can safely comment out the use of getticks. It's a counter that >> counts the number of CPU cycles. We used it for timing small bits of code >> that are called often and you need a very low overhead timing call. Not >> really sure why those unused references were left in! I think those calls >> were inserted manually when someone was actively working on optimizing that >> bit of code. >> >> --Subutai >> >> >> On Wed, Aug 21, 2013 at 7:57 PM, Matt Keith <[email protected]> wrote: >> >>> Progress, here is the result of the run_tests script: >>> === 216 passed, 4 skipped in 3149.59 seconds === >>> >>> No more errors! I am doing a clean build now and will test some more in >>> the morning. I have committed all of my files and changes to my keithcom >>> github fork. >>> >>> Does anyone know what the getticks function is used for in the >>> cycle_counter.hpp file? I was working on getting that to work on the Pi, >>> when I noticed that the only place it was referenced was in the Cells4.cpp >>> file. That file calls the function and stores the result in a local >>> variable and then never references it anywhere. Is there some funny >>> business going on that I am not seeing, or is this just left over from some >>> old code? For now, I have just commented out the references in the >>> Cells4.cpp file. Please let me know if this will cause a problem somewhere. >>> >>> Thanks, >>> >>> Matt >>> >>> On Aug 21, 2013, at 9:05 AM, Georgios Pierris <[email protected]> >>> wrote: >>> >>> Uhmm.. I see. I had some similar issues when I wanted to distribute a >>> virtual machine and had to shrink it. There were a couple of ways to do it, >>> not sure if applicable in your case. >>> >>> One was to use dd and write a massive file full of zeros. That will >>> pretty much take up all the free unallocated space on your sd card which >>> had any values from past write operations. Then you simply delete it. Or >>> you can use a tool called zerofree. >>> >>> After that, you end up with many zeros in your clone, which compression >>> algorithms exploit and you will end up with a much smaller tar.gz or >>> whatever you decide to use. >>> >>> Cheers, >>> Georgios >>> >>> >>> On Wed, Aug 21, 2013 at 3:45 PM, Matt Keith <[email protected]> wrote: >>> >>>> Yes, we should be able to just distribute an image file of the full >>>> system. I will need to do a clean build as my current system has a bunch >>>> of additional items installed as I was investigating the build process. >>>> Also, I would like to look into ways to shrink the image as I am currently >>>> on an 8 GB sd card to allow space for the OS, the 1 GB swap file and >>>> software. >>>> >>>> I'm going to look into my changes in the cycle_counter.hpp file to see >>>> if that is causing the seg fault. >>>> >>>> The quest continues . . . >>>> >>>> Matt >>>> >>>> On Aug 21, 2013, at 5:43 AM, Georgios Pierris <[email protected]> >>>> wrote: >>>> >>>> Great Matt! I am afraid I cannot help you with these errors, but I just >>>> wanted to know, is it safe to assume that once you have compiled everything >>>> you could "distribute" your sd clone with dd so that anyone else could get >>>> started within minutes? >>>> >>>> Cheers, >>>> Georgios >>>> >>>> >>>> On Wed, Aug 21, 2013 at 3:19 AM, Matt Keith <[email protected]> wrote: >>>> >>>>> Success! Thank you so much Georgios for the helpful information. >>>>> >>>>> Ok, so the entire build finally completed on the Raspberry Pi after >>>>> nearly 6 hours! I had to increase the swap size to 1 GB to get it to >>>>> complete without blowing up. Developing on the Pi really brings back >>>>> memories of this: >>>>> http://imgs.xkcd.com/comics/compiling.png >>>>> >>>>> That's the good news. Back to reality, it doesn't pass all of the >>>>> tests. >>>>> >>>>> The first thing that I noticed is that the >>>>> external/linux32/lib/python/site-packages contains some .so files. I was >>>>> just using the files from the linux64 dir to try and get things to build, >>>>> but we need to get arm versions of the .so files in matplotlib, Crypto, >>>>> and >>>>> PIL. So I used pip to install matplotlib and PIL and copied the dirs from >>>>> the system location to the nupic/external dirs. Then I installed >>>>> python-crypto with apt-get and copied the Crypto dir over. Is this a >>>>> valid >>>>> way to get the libs? Does anyone know how the shared objects were built >>>>> for the current linux64 and darwin64 dirs? >>>>> >>>>> When I run the nupic/bin/run_tests.py it shows: >>>>> === 6 failed, 210 passed, 4 skipped in 275.98 seconds === >>>>> >>>>> The htmtest binary dies with a seg fault right after 'Adding a PyNode >>>>> region...'. And the testeverything app shows: >>>>> Found 40 test sets >>>>> WARNING: Caught exception: This exception should get caught. >>>>> ======= Tests for TesterTest ============== >>>>> Total tests: 10, Failures: 4, Disabled : 0 >>>>> WARNING: Critical failure ocurred >>>>> <snip> >>>>> FAIL Integer test, should fail (line: 50) >>>>> Expected result: 1 >>>>> Actual result: 0 >>>>> FAIL Double test, should fail (line: 51) >>>>> Expected result: 23.42 >>>>> Actual result: 23.421 >>>>> FAIL String test, should fail (line: 52) >>>>> Expected result: Numenta >>>>> Actual result: Numenta >>>>> <snip> >>>>> ERROR: Invalid NTA_BasicType -1 used in array constructor >>>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 59] >>>>> ERROR: allocateBuffer -- buffer already set. Use releaseBuffer first >>>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 80] >>>>> ERROR: allocateBuffer -- buffer already set. Use releaseBuffer first >>>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 80] >>>>> ERROR: setBuffer -- buffer already set. Use releaseBuffer first >>>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 97] >>>>> ======= Tests for ArrayTest ============== >>>>> Total tests: 72, Failures: 0, Disabled : 0 >>>>> All tests passed >>>>> <snip> >>>>> ======= Tests for BasicTypeTest ============== >>>>> Total tests: 47, Failures: 0, Disabled : 0 >>>>> All tests passed >>>>> <snip> >>>>> ======= Tests for BufferTest ============== >>>>> Total tests: 94, Failures: 0, Disabled : 0 >>>>> All tests passed >>>>> >>>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>>> line 95] >>>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>>> line 95] >>>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>>> line 95] >>>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>>> line 95] >>>>> ERROR: Unable to add item '0' to collection because it already exists >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>>> ERROR: Unable to add item '1' to collection because it already exists >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>>> ERROR: Unable to add item '2' to collection because it already exists >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>>> ERROR: No item named '4' in collection >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>>> ERROR: No item named '1' in collection >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>>> ERROR: Unable to add item '0' to collection because it already exists >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>>> ERROR: Unable to add item '1' to collection because it already exists >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>>> ERROR: Unable to add item '2' to collection because it already exists >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>>> ERROR: No item named '1' in collection >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>>> ERROR: No item named '0' in collection >>>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>>> ======= Tests for CollectionTest ============== >>>>> Total tests: 199, Failures: 0, Disabled : 0 >>>>> All tests passed >>>>> <snip> >>>>> ======= Tests for CondProbTableTest ============== >>>>> Total tests: 115, Failures: 0, Disabled : 0 >>>>> All tests passed >>>>> <snip> >>>>> ======= Tests for DenseTensorUnitTest ============== >>>>> Total tests: 0, Failures: 0, Disabled : 0 >>>>> All tests passed >>>>> >>>>> ERROR: Attempt to get count from dimensions [unspecified] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57] >>>>> ERROR: Bad request for dimension 0 on [unspecified] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>>> ERROR: Invalid coordinate [1 2] for Dimensions [unspecified] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 171] >>>>> ERROR: Attempt to get count from dimensions [unspecified] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57] >>>>> ERROR: Bad request for dimension 0 on [unspecified] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>>> ERROR: Invalid coordinate index 0 of 0 is too large for region >>>>> dimensions [dontcare] [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 189] >>>>> ERROR: Attempt to get count from dimensions [dontcare] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57] >>>>> ERROR: Invalid coordinate index 0 of 1 is too large for region >>>>> dimensions [1 0] (invalid) [/home/pi/nupic/nta/ntypes/Dimensions.cpp line >>>>> 189] >>>>> ERROR: Attempt to get count from invalid dimensions [1 0] (invalid) >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 62] >>>>> ERROR: Bad request for dimension 2 on [1 0] (invalid) >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>>> ERROR: Bad request for dimension 2 on [2 3] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>>> ERROR: Bad request for dimension 2 on [2 5] >>>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>>> ======= Tests for DimensionsTest ============== >>>>> Total tests: 205, Failures: 0, Disabled : 0 >>>>> All tests passed >>>>> >>>>> Let me know if you want the full output. Any idea of what piece might >>>>> be broken? >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> On Aug 19, 2013, at 1:29 PM, Georgios Pierris <[email protected]> >>>>> wrote: >>>>> >>>>> Hello Matt, >>>>> >>>>> have a look on my comment in https://issues.numenta.org/browse/NPC-143 >>>>> regarding >>>>> the redefinition on swig. >>>>> >>>>> Cheers, >>>>> Georgios >>>>> >>>>> >>>>> On Mon, Aug 19, 2013 at 8:25 PM, Matt Keith <[email protected]>wrote: >>>>> >>>>>> Mark, >>>>>> >>>>>> Your linux32 compiled .a files do not work in the ARM environment, so >>>>>> I think that we will need to create a new external/linux32arm dir to >>>>>> support that platform. I'll try to get the build working with that >>>>>> change. >>>>>> >>>>>> I didn't remove any of the grouper files, I just commented out the >>>>>> guts of all of the Grouper.cpp methods. However, the problem was with >>>>>> the >>>>>> gcc 4.6 compiler and not the code, so it just failed on some different >>>>>> files later in the build. Upgrading to gcc 4.7 fixed the issue. I'll >>>>>> add >>>>>> my +1 to removing the old code, but I don't think that it is an issue for >>>>>> raspberry pi anymore. >>>>>> >>>>>> I spoke a little too soon saying that the build completed in my last >>>>>> message. There was an error with apr.h buried in the build output that I >>>>>> missed and that is why none of the test scripts were created. I have >>>>>> corrected that error but now I am getting the following error with swig. >>>>>> >>>>>> make[2]: Entering directory >>>>>> `/tmp/ntabuild/lang/py/bindings/engine_internal' >>>>>> /bin/bash ../../../../libtool --tag=CXX --mode=compile g++ >>>>>> -DHAVE_CONFIG_H -I. -I/home/pi/nupic/lang/py/bindings/engine_internal >>>>>> -I../../../../build_system/unix -I/usr/include/python2.7 >>>>>> -I/home/pi/nupic >>>>>> -I/home/pi/nupic/external/linux32/include -DNUPIC2 -I/home/pi/nupic >>>>>> -I/home/pi/nupic/external/common/include >>>>>> -I/home/pi/nupic/external/linux32/include -fvisibility=hidden >>>>>> -DNTA_INTERNAL -g -DNTA_ASSERTIONS_ON -fPIC -DPIC -ffloat-store -D_PY27 >>>>>> -DNTA_PLATFORM_linux32 -Wall -Wreturn-type -Wunused-variable >>>>>> -Wno-deprecated -DBOOST_NO_WREGEX -DNTA_INSTRUMENT_ON >>>>>> -DNTA_PYTHON_SUPPORT=2.7 -DNTAPY_INCLUDE="<python2.7/" >>>>>> -DNTAPY_PACKAGES="\"lib/python2.7/site-packages\"" -I/usr/include >>>>>> -I/lib/python2.7/site-packages/numpy/core/include -O1 -falign-functions >>>>>> -DSWIG_TYPE_TABLE=_nupic_engine_internal -DNUPIC2 -MT >>>>>> _engine_internal_la-engine_internal_py.lo -MD -MP -MF >>>>>> .deps/_engine_internal_la-engine_internal_py.Tpo -c -o >>>>>> _engine_internal_la-engine_internal_py.lo `test -f >>>>>> 'engine_internal_py.cpp' >>>>>> || echo >>>>>> '/home/pi/nupic/lang/py/bindings/engine_internal/'`engine_internal_py.cpp >>>>>> libtool: compile: g++ -DHAVE_CONFIG_H -I. >>>>>> -I/home/pi/nupic/lang/py/bindings/engine_internal >>>>>> -I../../../../build_system/unix -I/usr/include/python2.7 -I/home/pi/nupic >>>>>> -I/home/pi/nupic/external/linux32/include -DNUPIC2 -I/home/pi/nupic >>>>>> -I/home/pi/nupic/external/common/include >>>>>> -I/home/pi/nupic/external/linux32/include -fvisibility=hidden >>>>>> -DNTA_INTERNAL -g -DNTA_ASSERTIONS_ON -fPIC -DPIC -ffloat-store -D_PY27 >>>>>> -DNTA_PLATFORM_linux32 -Wall -Wreturn-type -Wunused-variable >>>>>> -Wno-deprecated -DBOOST_NO_WREGEX -DNTA_INSTRUMENT_ON >>>>>> -DNTA_PYTHON_SUPPORT=2.7 "-DNTAPY_INCLUDE=<python2.7/" >>>>>> -DNTAPY_PACKAGES=\"lib/python2.7/site-packages\" -I/usr/include >>>>>> -I/lib/python2.7/site-packages/numpy/core/include -O1 -falign-functions >>>>>> -DSWIG_TYPE_TABLE=_nupic_engine_internal -DNUPIC2 -MT >>>>>> _engine_internal_la-engine_internal_py.lo -MD -MP -MF >>>>>> .deps/_engine_internal_la-engine_internal_py.Tpo -c >>>>>> engine_internal_py.cpp >>>>>> -fPIC -DPIC -o .libs/_engine_internal_la-engine_internal_py.o >>>>>> engine_internal_py.cpp:4455:22: error: redefinition of 'struct >>>>>> swig::traits<unsigned int>' >>>>>> engine_internal_py.cpp:4378:22: error: previous definition of 'struct >>>>>> swig::traits<unsigned int>' >>>>>> engine_internal_py.cpp:4459:23: error: redefinition of 'struct >>>>>> swig::traits_asval<unsigned int>' >>>>>> engine_internal_py.cpp:4382:23: error: previous definition of 'struct >>>>>> swig::traits_asval<unsigned int>' >>>>>> engine_internal_py.cpp:4465:23: error: redefinition of 'struct >>>>>> swig::traits_from<unsigned int>' >>>>>> engine_internal_py.cpp:4388:23: error: previous definition of 'struct >>>>>> swig::traits_from<unsigned int>' >>>>>> make[2]: *** [_engine_internal_la-engine_internal_py.lo] Error 1 >>>>>> >>>>>> I have not used swig before. Are there any swig experts out there >>>>>> that can point me the right direction? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> On Aug 19, 2013, at 2:47 AM, Marek Otahal <[email protected]> >>>>>> wrote: >>>>>> >>>>>> Hi Matt, >>>>>> >>>>>> >>>>>> On Sun, Aug 18, 2013 at 8:33 AM, Matt Keith <[email protected]>wrote: >>>>>> >>>>>>> Thank you everyone for the feedback. >>>>>>> >>>>>>> Marek, yes I'll have to take a look at the linux32 support you have >>>>>>> submitted. Won't there be issues with those .a files working on arm >>>>>>> instead of x86? Should we create a linux32arm folder as well, or can we >>>>>>> remove all dependancies on static libraries? >>>>>>> >>>>>> I was wondering that too. Could you test, take the .a files from my >>>>>> linux32 branch and place them in your ARM environment? >>>>>> (I think it will be a prob and we'll need another platform to >>>>>> support) >>>>>> >>>>>> about removing all the static deps: I'm not sure now, as there are >>>>>> tickets like linux64/darwin64-environment, ... But before I was told we >>>>>> want to keep nupic as much self-contined as possible.(but building some >>>>>> of >>>>>> the libs might be tricky, that's why the precompiled archives .a are >>>>>> present) Maybe Scott can explain? >>>>>> >>>>>> PS: did you succeed removing the old Grouper classes, as Subutai >>>>>> mentioned here? I tried just rm the related files, but it was quite >>>>>> connected to other code too (so I wonder if that code is obsolete too, >>>>>> and >>>>>> can be removed, or need be reworked) >>>>>> >>>>>> Regards, Mark >>>>>> >>>>>> >>>>>> -- >>>>>> Marek Otahal :o) >>>>>> _______________________________________________ >>>>>> nupic mailing list >>>>>> [email protected] >>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> nupic mailing list >>>>>> [email protected] >>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> nupic mailing list >>>>> [email protected] >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> nupic mailing list >>>>> [email protected] >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>> >>>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > > -- > Marek Otahal :o) > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
