Great work Matt! Thanks. Can you make a PR and get this stuff upstream once you think the branch is ready? thank you, Mark
On Thu, Aug 22, 2013 at 6:08 AM, Subutai Ahmad <[email protected]> wrote: > > That's excellent! > > You can safely comment out the use of getticks. It's a counter that counts > the number of CPU cycles. We used it for timing small bits of code that are > called often and you need a very low overhead timing call. Not really sure > why those unused references were left in! I think those calls were inserted > manually when someone was actively working on optimizing that bit of code. > > --Subutai > > > On Wed, Aug 21, 2013 at 7:57 PM, Matt Keith <[email protected]> wrote: > >> Progress, here is the result of the run_tests script: >> === 216 passed, 4 skipped in 3149.59 seconds === >> >> No more errors! I am doing a clean build now and will test some more in >> the morning. I have committed all of my files and changes to my keithcom >> github fork. >> >> Does anyone know what the getticks function is used for in the >> cycle_counter.hpp file? I was working on getting that to work on the Pi, >> when I noticed that the only place it was referenced was in the Cells4.cpp >> file. That file calls the function and stores the result in a local >> variable and then never references it anywhere. Is there some funny >> business going on that I am not seeing, or is this just left over from some >> old code? For now, I have just commented out the references in the >> Cells4.cpp file. Please let me know if this will cause a problem somewhere. >> >> Thanks, >> >> Matt >> >> On Aug 21, 2013, at 9:05 AM, Georgios Pierris <[email protected]> wrote: >> >> Uhmm.. I see. I had some similar issues when I wanted to distribute a >> virtual machine and had to shrink it. There were a couple of ways to do it, >> not sure if applicable in your case. >> >> One was to use dd and write a massive file full of zeros. That will >> pretty much take up all the free unallocated space on your sd card which >> had any values from past write operations. Then you simply delete it. Or >> you can use a tool called zerofree. >> >> After that, you end up with many zeros in your clone, which compression >> algorithms exploit and you will end up with a much smaller tar.gz or >> whatever you decide to use. >> >> Cheers, >> Georgios >> >> >> On Wed, Aug 21, 2013 at 3:45 PM, Matt Keith <[email protected]> wrote: >> >>> Yes, we should be able to just distribute an image file of the full >>> system. I will need to do a clean build as my current system has a bunch >>> of additional items installed as I was investigating the build process. >>> Also, I would like to look into ways to shrink the image as I am currently >>> on an 8 GB sd card to allow space for the OS, the 1 GB swap file and >>> software. >>> >>> I'm going to look into my changes in the cycle_counter.hpp file to see >>> if that is causing the seg fault. >>> >>> The quest continues . . . >>> >>> Matt >>> >>> On Aug 21, 2013, at 5:43 AM, Georgios Pierris <[email protected]> >>> wrote: >>> >>> Great Matt! I am afraid I cannot help you with these errors, but I just >>> wanted to know, is it safe to assume that once you have compiled everything >>> you could "distribute" your sd clone with dd so that anyone else could get >>> started within minutes? >>> >>> Cheers, >>> Georgios >>> >>> >>> On Wed, Aug 21, 2013 at 3:19 AM, Matt Keith <[email protected]> wrote: >>> >>>> Success! Thank you so much Georgios for the helpful information. >>>> >>>> Ok, so the entire build finally completed on the Raspberry Pi after >>>> nearly 6 hours! I had to increase the swap size to 1 GB to get it to >>>> complete without blowing up. Developing on the Pi really brings back >>>> memories of this: >>>> http://imgs.xkcd.com/comics/compiling.png >>>> >>>> That's the good news. Back to reality, it doesn't pass all of the >>>> tests. >>>> >>>> The first thing that I noticed is that the >>>> external/linux32/lib/python/site-packages contains some .so files. I was >>>> just using the files from the linux64 dir to try and get things to build, >>>> but we need to get arm versions of the .so files in matplotlib, Crypto, and >>>> PIL. So I used pip to install matplotlib and PIL and copied the dirs from >>>> the system location to the nupic/external dirs. Then I installed >>>> python-crypto with apt-get and copied the Crypto dir over. Is this a valid >>>> way to get the libs? Does anyone know how the shared objects were built >>>> for the current linux64 and darwin64 dirs? >>>> >>>> When I run the nupic/bin/run_tests.py it shows: >>>> === 6 failed, 210 passed, 4 skipped in 275.98 seconds === >>>> >>>> The htmtest binary dies with a seg fault right after 'Adding a PyNode >>>> region...'. And the testeverything app shows: >>>> Found 40 test sets >>>> WARNING: Caught exception: This exception should get caught. >>>> ======= Tests for TesterTest ============== >>>> Total tests: 10, Failures: 4, Disabled : 0 >>>> WARNING: Critical failure ocurred >>>> <snip> >>>> FAIL Integer test, should fail (line: 50) >>>> Expected result: 1 >>>> Actual result: 0 >>>> FAIL Double test, should fail (line: 51) >>>> Expected result: 23.42 >>>> Actual result: 23.421 >>>> FAIL String test, should fail (line: 52) >>>> Expected result: Numenta >>>> Actual result: Numenta >>>> <snip> >>>> ERROR: Invalid NTA_BasicType -1 used in array constructor >>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 59] >>>> ERROR: allocateBuffer -- buffer already set. Use releaseBuffer first >>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 80] >>>> ERROR: allocateBuffer -- buffer already set. Use releaseBuffer first >>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 80] >>>> ERROR: setBuffer -- buffer already set. Use releaseBuffer first >>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 97] >>>> ======= Tests for ArrayTest ============== >>>> Total tests: 72, Failures: 0, Disabled : 0 >>>> All tests passed >>>> <snip> >>>> ======= Tests for BasicTypeTest ============== >>>> Total tests: 47, Failures: 0, Disabled : 0 >>>> All tests passed >>>> <snip> >>>> ======= Tests for BufferTest ============== >>>> Total tests: 94, Failures: 0, Disabled : 0 >>>> All tests passed >>>> >>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>> line 95] >>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>> line 95] >>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>> line 95] >>>> ERROR: CHECK FAILED: "index < vec_.size()" >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70] >>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp >>>> line 95] >>>> ERROR: Unable to add item '0' to collection because it already exists >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>> ERROR: Unable to add item '1' to collection because it already exists >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>> ERROR: Unable to add item '2' to collection because it already exists >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>> ERROR: No item named '4' in collection >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>> ERROR: No item named '1' in collection >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>> ERROR: Unable to add item '0' to collection because it already exists >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>> ERROR: Unable to add item '1' to collection because it already exists >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>> ERROR: Unable to add item '2' to collection because it already exists >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107] >>>> ERROR: No item named '1' in collection >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>> ERROR: No item named '0' in collection >>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127] >>>> ======= Tests for CollectionTest ============== >>>> Total tests: 199, Failures: 0, Disabled : 0 >>>> All tests passed >>>> <snip> >>>> ======= Tests for CondProbTableTest ============== >>>> Total tests: 115, Failures: 0, Disabled : 0 >>>> All tests passed >>>> <snip> >>>> ======= Tests for DenseTensorUnitTest ============== >>>> Total tests: 0, Failures: 0, Disabled : 0 >>>> All tests passed >>>> >>>> ERROR: Attempt to get count from dimensions [unspecified] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57] >>>> ERROR: Bad request for dimension 0 on [unspecified] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>> ERROR: Invalid coordinate [1 2] for Dimensions [unspecified] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 171] >>>> ERROR: Attempt to get count from dimensions [unspecified] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57] >>>> ERROR: Bad request for dimension 0 on [unspecified] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>> ERROR: Invalid coordinate index 0 of 0 is too large for region >>>> dimensions [dontcare] [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 189] >>>> ERROR: Attempt to get count from dimensions [dontcare] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57] >>>> ERROR: Invalid coordinate index 0 of 1 is too large for region >>>> dimensions [1 0] (invalid) [/home/pi/nupic/nta/ntypes/Dimensions.cpp line >>>> 189] >>>> ERROR: Attempt to get count from invalid dimensions [1 0] (invalid) >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 62] >>>> ERROR: Bad request for dimension 2 on [1 0] (invalid) >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>> ERROR: Bad request for dimension 2 on [2 3] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>> ERROR: Bad request for dimension 2 on [2 5] >>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78] >>>> ======= Tests for DimensionsTest ============== >>>> Total tests: 205, Failures: 0, Disabled : 0 >>>> All tests passed >>>> >>>> Let me know if you want the full output. Any idea of what piece might >>>> be broken? >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> On Aug 19, 2013, at 1:29 PM, Georgios Pierris <[email protected]> >>>> wrote: >>>> >>>> Hello Matt, >>>> >>>> have a look on my comment in https://issues.numenta.org/browse/NPC-143 >>>> regarding >>>> the redefinition on swig. >>>> >>>> Cheers, >>>> Georgios >>>> >>>> >>>> On Mon, Aug 19, 2013 at 8:25 PM, Matt Keith <[email protected]> wrote: >>>> >>>>> Mark, >>>>> >>>>> Your linux32 compiled .a files do not work in the ARM environment, so >>>>> I think that we will need to create a new external/linux32arm dir to >>>>> support that platform. I'll try to get the build working with that >>>>> change. >>>>> >>>>> I didn't remove any of the grouper files, I just commented out the >>>>> guts of all of the Grouper.cpp methods. However, the problem was with the >>>>> gcc 4.6 compiler and not the code, so it just failed on some different >>>>> files later in the build. Upgrading to gcc 4.7 fixed the issue. I'll add >>>>> my +1 to removing the old code, but I don't think that it is an issue for >>>>> raspberry pi anymore. >>>>> >>>>> I spoke a little too soon saying that the build completed in my last >>>>> message. There was an error with apr.h buried in the build output that I >>>>> missed and that is why none of the test scripts were created. I have >>>>> corrected that error but now I am getting the following error with swig. >>>>> >>>>> make[2]: Entering directory >>>>> `/tmp/ntabuild/lang/py/bindings/engine_internal' >>>>> /bin/bash ../../../../libtool --tag=CXX --mode=compile g++ >>>>> -DHAVE_CONFIG_H -I. -I/home/pi/nupic/lang/py/bindings/engine_internal >>>>> -I../../../../build_system/unix -I/usr/include/python2.7 -I/home/pi/nupic >>>>> -I/home/pi/nupic/external/linux32/include -DNUPIC2 -I/home/pi/nupic >>>>> -I/home/pi/nupic/external/common/include >>>>> -I/home/pi/nupic/external/linux32/include -fvisibility=hidden >>>>> -DNTA_INTERNAL -g -DNTA_ASSERTIONS_ON -fPIC -DPIC -ffloat-store -D_PY27 >>>>> -DNTA_PLATFORM_linux32 -Wall -Wreturn-type -Wunused-variable >>>>> -Wno-deprecated -DBOOST_NO_WREGEX -DNTA_INSTRUMENT_ON >>>>> -DNTA_PYTHON_SUPPORT=2.7 -DNTAPY_INCLUDE="<python2.7/" >>>>> -DNTAPY_PACKAGES="\"lib/python2.7/site-packages\"" -I/usr/include >>>>> -I/lib/python2.7/site-packages/numpy/core/include -O1 -falign-functions >>>>> -DSWIG_TYPE_TABLE=_nupic_engine_internal -DNUPIC2 -MT >>>>> _engine_internal_la-engine_internal_py.lo -MD -MP -MF >>>>> .deps/_engine_internal_la-engine_internal_py.Tpo -c -o >>>>> _engine_internal_la-engine_internal_py.lo `test -f >>>>> 'engine_internal_py.cpp' >>>>> || echo >>>>> '/home/pi/nupic/lang/py/bindings/engine_internal/'`engine_internal_py.cpp >>>>> libtool: compile: g++ -DHAVE_CONFIG_H -I. >>>>> -I/home/pi/nupic/lang/py/bindings/engine_internal >>>>> -I../../../../build_system/unix -I/usr/include/python2.7 -I/home/pi/nupic >>>>> -I/home/pi/nupic/external/linux32/include -DNUPIC2 -I/home/pi/nupic >>>>> -I/home/pi/nupic/external/common/include >>>>> -I/home/pi/nupic/external/linux32/include -fvisibility=hidden >>>>> -DNTA_INTERNAL -g -DNTA_ASSERTIONS_ON -fPIC -DPIC -ffloat-store -D_PY27 >>>>> -DNTA_PLATFORM_linux32 -Wall -Wreturn-type -Wunused-variable >>>>> -Wno-deprecated -DBOOST_NO_WREGEX -DNTA_INSTRUMENT_ON >>>>> -DNTA_PYTHON_SUPPORT=2.7 "-DNTAPY_INCLUDE=<python2.7/" >>>>> -DNTAPY_PACKAGES=\"lib/python2.7/site-packages\" -I/usr/include >>>>> -I/lib/python2.7/site-packages/numpy/core/include -O1 -falign-functions >>>>> -DSWIG_TYPE_TABLE=_nupic_engine_internal -DNUPIC2 -MT >>>>> _engine_internal_la-engine_internal_py.lo -MD -MP -MF >>>>> .deps/_engine_internal_la-engine_internal_py.Tpo -c engine_internal_py.cpp >>>>> -fPIC -DPIC -o .libs/_engine_internal_la-engine_internal_py.o >>>>> engine_internal_py.cpp:4455:22: error: redefinition of 'struct >>>>> swig::traits<unsigned int>' >>>>> engine_internal_py.cpp:4378:22: error: previous definition of 'struct >>>>> swig::traits<unsigned int>' >>>>> engine_internal_py.cpp:4459:23: error: redefinition of 'struct >>>>> swig::traits_asval<unsigned int>' >>>>> engine_internal_py.cpp:4382:23: error: previous definition of 'struct >>>>> swig::traits_asval<unsigned int>' >>>>> engine_internal_py.cpp:4465:23: error: redefinition of 'struct >>>>> swig::traits_from<unsigned int>' >>>>> engine_internal_py.cpp:4388:23: error: previous definition of 'struct >>>>> swig::traits_from<unsigned int>' >>>>> make[2]: *** [_engine_internal_la-engine_internal_py.lo] Error 1 >>>>> >>>>> I have not used swig before. Are there any swig experts out there >>>>> that can point me the right direction? >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> On Aug 19, 2013, at 2:47 AM, Marek Otahal <[email protected]> >>>>> wrote: >>>>> >>>>> Hi Matt, >>>>> >>>>> >>>>> On Sun, Aug 18, 2013 at 8:33 AM, Matt Keith <[email protected]>wrote: >>>>> >>>>>> Thank you everyone for the feedback. >>>>>> >>>>>> Marek, yes I'll have to take a look at the linux32 support you have >>>>>> submitted. Won't there be issues with those .a files working on arm >>>>>> instead of x86? Should we create a linux32arm folder as well, or can we >>>>>> remove all dependancies on static libraries? >>>>>> >>>>> I was wondering that too. Could you test, take the .a files from my >>>>> linux32 branch and place them in your ARM environment? >>>>> (I think it will be a prob and we'll need another platform to support) >>>>> >>>>> about removing all the static deps: I'm not sure now, as there are >>>>> tickets like linux64/darwin64-environment, ... But before I was told we >>>>> want to keep nupic as much self-contined as possible.(but building some of >>>>> the libs might be tricky, that's why the precompiled archives .a are >>>>> present) Maybe Scott can explain? >>>>> >>>>> PS: did you succeed removing the old Grouper classes, as Subutai >>>>> mentioned here? I tried just rm the related files, but it was quite >>>>> connected to other code too (so I wonder if that code is obsolete too, and >>>>> can be removed, or need be reworked) >>>>> >>>>> Regards, Mark >>>>> >>>>> >>>>> -- >>>>> Marek Otahal :o) >>>>> _______________________________________________ >>>>> nupic mailing list >>>>> [email protected] >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> nupic mailing list >>>>> [email protected] >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>> >>>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Marek Otahal :o)
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
