Re: [Cin] Cin Digest, Vol 51, Issue 40

2023-01-22 Thread Richard Nolde via Cin

Regarding memory leaks:

1. Re:  Complete system hangs (likely out of memory)
   (Stefan de Konink)
2. Re:  Complete system hangs (likely out of memory)
   (Stefan de Konink)
3. Re:  Complete system hangs (likely out of memory)
   (Andrew Randrianasulu)
The following tool has been instrumental in finding memory leaks and/or 
peaks of usage in a large graphically intensive program that I have 
helped another developer with from time to time.


https://apps.kde.org/heaptrack/
https://github.com/KDE/heaptrack

It comes with text and GUI analyzer tools to allow you to see which 
functions allocate/release/leak memory during the run of a program and 
it does not slow down the execution nearly as much as valgrind. It uses 
a replacement version of malloc and the LD_PRELOAD option to redirect 
system calls into its own instrumented versions of memory related system 
calls.  Originally written by Millan Wolff in Berlin, it has now become 
a part of the KDE development environment but it does not require you to 
run a full KDE desktop. I run in with XFCE on Fedora 36 without any 
problems.


Richard Nolde

--
Cin mailing list
Cin@lists.cinelerra-gg.org
https://lists.cinelerra-gg.org/mailman/listinfo/cin


Re: [Cin] PreRoll in Cin Digest, Vol 43, Issue 13,

2022-05-04 Thread Richard Nolde via Cin

  
  
Phyllis et al,
  
  While I have not tested the current state of the git repository, I
  thought I might add a few comments about testing performance in
  general from the perspective of a retired systems programmer. I am
  a fan of Cinelerra-GG and use it for burning bluray disks so I
  hope to be able to test the new code before long.
  
  I realize that the posts to which I am referring are not intended
  as exhaustive, definitive test suites, but it can be useful to
  know how the results can be affected. On most operating systems,
  memory requested by an application is not returned to the system
  wide free pool when the application releases it with a call to
  free or delete. You can see this by running a memory intensive
  application in one window and the gnome-system-monitor (or
  equivalent) in another window. Unless there is a great deal of
  demand for free memory by other applications, this memory will
  remain assigned to the first application until that application is
  closed. Therefore, each test should be run from a fresh invocation
  of the test application. Unfortunately, this doesn't guarantee
  that the file system cache is flushed.
  
  Any settings within a given application, eg cache or pre-roll,
  will need to be measured in a controlled environment which means
  rebooting the computer to clear the file system file cache.
  Otherwise, any request for data that has been previously read may
  be satisfied from the operating systems file cache regardless of
  the settings in the application. That said, the effect of the file
  system cache loading will be most or exclusively felt in short
  tests that fit completely or mostly within the file system cache.
  The tests should be run with as few other applications loaded as
  possible to minimize CPU contention and disk access. If there is a
  way to run the test from the command line, this will eliminate
  additional video card specific rendering issues, by which I mean
  presenting the images on the screen. I am not referring to using
  GPU's used to render the video to the files on disk that will be
  read during playback. Of course, if what you are trying to measure
  is the onscreen presentation rate, then a command line test is not
  appropriate.
  
  The terminal commands free and top can be used to see how much
  memory is being allocated to caches. If you run these before and
  after a test, you should see an increase in the memory allocated
  to the file caches. 
  
  You can use the command: cat /proc/meminfo to see how much memory
  is being allocated to file system caches. Note: these values are
  for an older laptop with only 8 GB of physical ram and an 8 GB
  physical swap partition.
  
  cat /proc/meminfo
  MemTotal:    7802112 kB
MemFree: 2199052 kB
MemAvailable:    5405988 kB
Buffers:  175436 kB
Cached:  3361856 kB
SwapCached:    0 kB
Active:  1470940 kB
Inactive:    3556264 kB
Active(anon):   4204 kB
Inactive(anon):  1616348 kB
Active(file):    1466736 kB
Inactive(file):  1939916 kB
  
swapon
NAME   TYPE  SIZE USED PRIO
/dev/sda3  partition 7.9G   0B   -2
/dev/zram0 partition 7.4G   0B  100

See the URL below for a description of the zram0 swap partition.
https://www.kernel.org/doc/html/v5.3/admin-guide/blockdev/zram.html
  
  It goes without saying that any use of swap space will radically
  degrade performance unless that swap space is on a solid state
  drive and the swap partition is quite generous in size. Recent
  versions of Fedora, ie 33 and newer, seem to favor a compressed
  swap space and/or using memory compression, which can have a large
  impact on memory intensive operations. My recommendation is to use
  a dedicated, on disk (SSD) swap partition that is twice the size
  of physical memory or larger depending on the memory requirements
  of your application. I use the Fotoxx graphics application to make
  very large panoramas and have frequently used up to 40 GB of swap
  space on a system with 16 GB of physical ram. Cinerlerra is
  nowhere near that memory hungry, but it is essential to avoid
  running into situations where the kernel will have to evict code
  pages or data from the cache. The output of the swapon
  command indicates that the compressed zram0 block device has a
  much higher priority than the physical disk based swap partition.
  This can negatively impact file caching if your on disk swap
  partition is fast and your memory is older, slower memory. It may
  be worth trying to limit the size of