2013/8/24 Peter Prettenhofer <[email protected]>: > the SGD code assumes that the data is in one continouse memory segment - > does this hold for memory mapped arrays? > What if you create a memory mapped array that is larger than the available > memory?
Yes, that holds, and an mmap that is larger than the available physical memory (RAM + disk where the mapping is done) should fail with ENOMEM, I think. But note that the size of the mapped design matrix is 4e9 elements; that's more than a 32-bit (or signed 31-bit) integer can index. I'm not sure if it is the problem, but the weight vector code is using such indices. (Strangely, though, neither n_samples nor n_features exceeds this limit). > there are no test cases that cover input of memory mapped arrays -- if we > want to support those I strongly recommend to add them to the common test > suite. > > > 2013/8/24 Vlad Niculae <[email protected]> >> >> Is it maybe related to the OS, as it seems that the problem is with >> opening the memmapped file? >> >> Vlad >> >> >> On Sat, Aug 24, 2013 at 1:52 PM, Olivier Grisel <[email protected]> >> wrote: >>> >>> Sounds like a serious bug, could you please open an issue on github? >>> >>> -- >>> Olivier >>> >>> >>> ------------------------------------------------------------------------------ >>> Introducing Performance Central, a new site from SourceForge and >>> AppDynamics. Performance Central is your source for news, insights, >>> analysis and resources for efficient Application Performance Management. >>> Visit us today! >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> >> >> >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Scikit-learn-general mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> > > > > -- > Peter Prettenhofer > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Scikit-learn-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > -- Lars Buitinck Scientific programmer, ILPS University of Amsterdam ------------------------------------------------------------------------------ Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
