Ha, On 11/18/2016 01:36 PM, Austin S. Hemmelgarn wrote: > On 2016-11-17 16:08, Hans van Kranenburg wrote: >> On 11/17/2016 08:27 PM, Austin S. Hemmelgarn wrote: >>> On 2016-11-17 13:51, Hans van Kranenburg wrote: >> But, the fun with visualizations of data is that you learn whether they >> just work(tm) or don't as soon as you see them. Mathematical or >> algorithmic beauty is not always a good recipe for beauty as seen by the >> human eye. >> >> So, let's gather a bunch of ideas which we can try out and then observe >> the result. >> >> Before doing so, I'm going to restructure the code a bit more so I can >> write another script in the same directory, just doing import heatmap >> and calling a few functions in there to quickly try stuff, bypassing the >> normal cli api. >> >> Also, the png writing handling is now done by some random png library >> that I found, which requires me to build (or copy/resize) an entire >> pixel grid in memory, explicitely listing all pixel values, which is a >> bit of a memory hog for bigger pictures, so I want to see if something >> can be done there also. > I haven't had a chance to look at the code yet, but do you have an > option to control how much data a pixel represents? On a multi TB > filesystem for example, you may not care about exact data, just an > overall view of the data, in which case making each pixel represent a > larger chunk of data (and thus reducing the resolution of the image) > would almost certainly save some memory on big filesystems.
--order, which defines the hilbert curve order. Example: for a 238GiB filesystem, when specifying --order 7, then 2**7 = 128, so 128x128 = 16384 pixels, which means that a single one represents ~16MiB when --size > --order, the image simply gets scaled up. When not specifying --order, a number gets chosen automatically with which bytes per pixel is closest to 32MiB. When size is not specified, it's 10, or same as order if order is greater than 10. Now this output should make sense: -# ./heatmap.py /mnt/238GiB max_id 1 num_devices 1 fsid ed108358-c746-4e76-a071-3820d423a99d nodesize 16384 sectorsize 4096 clone_alignment 4096 scope filesystem curve hilbert order 7 size 10 pngfile fsid_ed10a358-c846-4e76-a071-3821d423a99d_at_1479473532.png grid height 128 width 128 total_bytes 255057723392 bytes_per_pixel 15567488.0 pixels 16384 -# ./heatmap.py /mnt/40TiB max_id 2 num_devices 2 fsid 9bc9947e-070f-4bbc-872e-49b2a39b3f7b nodesize 16384 sectorsize 4096 clone_alignment 4096 scope filesystem curve hilbert order 10 size 10 pngfile /home/beheer/heatmap/generated/fsid_9bd9947e-070f-4cbc-8e2e-49b3a39b8f7b_at_1479473950.png grid height 1024 width 1024 total_bytes 46165378727936 bytes_per_pixel 44026736.0 pixels 1048576 -- Hans van Kranenburg -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
