Yes that would be a better solution. I think the wb_command -cifti-restrict-dense-map might work.
Matt. From: Reza Rajimehr <rajim...@gmail.com<mailto:rajim...@gmail.com>> Date: Thursday, October 25, 2018 at 3:16 AM To: Matt Glasser <glass...@wustl.edu<mailto:glass...@wustl.edu>> Cc: Timothy Coalson <tsc...@mst.edu<mailto:tsc...@mst.edu>>, "hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> Subject: Re: [HCP-Users] Question about dense functional connectome We tried to open *.dconn.nii in Matlab using ft_read_cifti on an Ubuntu system with 48 GB RAM. It used 48 GB RAM and 18 GB (out of 60 GB) swap, then Matlab gave out of memory error! Will try to use a system with even higher RAM. Tim: We have a list of 913 grayordinate voxels (we have their voxel numbers/indices). These voxels are somewhat distributed in gray matter. We are basically interested in the functional connectivity matrix just for these 913 voxels (i.e. a 913 * 913 matrix). Is it possible to use wb_command to select part of *.dconn.nii corresponding to the voxels of interest? Best, Reza On Thu, Oct 25, 2018 at 3:49 AM Glasser, Matthew <glass...@wustl.edu<mailto:glass...@wustl.edu>> wrote: Use swap space. Matt. From: <hcp-users-boun...@humanconnectome.org<mailto:hcp-users-boun...@humanconnectome.org>> on behalf of Timothy Coalson <tsc...@mst.edu<mailto:tsc...@mst.edu>> Date: Wednesday, October 24, 2018 at 4:57 PM To: Reza Rajimehr <rajim...@gmail.com<mailto:rajim...@gmail.com>> Cc: "hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> Subject: Re: [HCP-Users] Question about dense functional connectome The cifti format doesn't support that, because it is designed to allow use without loading the entire file into memory - there is no obvious file organization that would allow efficient loading of a full row from a single triangular file, when seek times are nontrivial (rotating disks). I don't believe we have these files in any other format. I'm also not sure how easy it would be to work on such a matrix in matlab. If wb_command is capable of doing the operations you want, it is usually possible for it to do them without loading the entire cifti file into memory at once. wb_view will also display maps from it without loading the entire file into memory. DDR4 has 16GB modules available at reasonable prices, which should allow recent computers with even 4 slots for memory to expand to 64GB. You should also consider the "high-end desktop" platforms (threadripper, lga2066, lga2011), which typically offer 8 memory slots, and processors with more cores. Tim On Wed, Oct 24, 2018 at 3:56 PM, Reza Rajimehr <rajim...@gmail.com<mailto:rajim...@gmail.com>> wrote: Hi, We are trying to unzip and load dense functional connectome (HCP_S900_820_rfMRI_MSMAll_groupPCA_d4500ROW_zcorr.zip or HCP_S900_820_rfMRI_MSMSulc_groupPCA_d4500ROW_zcorr.zip from https://www.humanconnectome.org/study/hcp-young-adult/article/announcing-release-of-s900-ptn-and-other-group-average-data) in Matlab on a system that has 32 GB RAM. This is of course impossible due to the fact that the files are ~33 GB in size. Due to matrix symmetry, it would be sufficient to store only the upper or lower triangular part of a functional connectivity matrix, reducing memory occupancy by about 50% (Loewe et al., 2016). Does HCP provide such files that could be memory-friendly? Best, Reza _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org<mailto:HCP-Users@humanconnectome.org> http://lists.humanconnectome.org/mailman/listinfo/hcp-users _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org<mailto:HCP-Users@humanconnectome.org> http://lists.humanconnectome.org/mailman/listinfo/hcp-users ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users