Here is a patch to do some basic cheap refmem counting/byte allocation counting,
does it look correct?
It produces output like this (after *ONLY* entering watch recordings):
before cmyth_proglist_create: Total Refs: 9 Total Bytes: 207
after cmyth_proglist_create: Total Refs: 10 Total Bytes: 227
before cmyth_proglist_get_all_recorded: Total Refs: 10 Total Bytes:
227
event thread started (pid 159)
after cmyth_proglist_get_all_recorded: Total Refs: 21790 Total Bytes:
763174
The refcounter takes 12 bytes per reference, so of the total usage of 763174
bytes, 261000 is the refcounter (or roughly 1/3 of the total memory used for
programs info).
Doing some testing it does appear to leak (it takes 21000 to load my recordings
but I have seem up to 5x that number-more testing to follow).
Also a "timestamp" takes 7-4 byte ints, and there are 6 timestamps per program
info so in my case 6 x 7 x 4 x 725 = 121,000 bytes for just the timestamps, so
if one were able to use "unsigned char" and if the structure will pack in that
case (I don't know what sort of alignment issues there are on a PPC, on i386
there would not be an issue, on IA64 slow kernel interrupts have to take care of
that), but if it would work that would reduced the timestamps to 1/4 of that.
Roger
Simon Hyde wrote:
On Fri, 30 May 2008, Roger Heflin wrote:
Hi,
Sorry for replying so late, kinda forgot about this thread... :)
Anyway, I just got a H3 wireless MVP and am seing the same problems
(not just when deleting). I'm up to 1900 movies/episodes on my MythTV
backend and moving around in the mvpmc menus is an excersice in
patience... selecting MythTV->Watch Recordings takes a long time to
end up in a usable state.
I am up to 720 and so far it is not that bad.
I have noticed that it takes 3-5 seconds to load the shows, I am
wondering if it
would be faster to read in from a binary file located on NFS that
contained
the show data, or put a binary file on NFS that at least contains the
more
troublesome data and retrieve it as necessary (when one selects show
info).
This sounds like a bit of a cludge, we'd be better off accessing the
data directly over SQL than requiring some extra server-side
functionality/files. If we had to resort to SQL for normal recordings
then we might want to make it optional. It's nice that at the moment SQL
access is only required for LiveTV.
From some looking around with the current method that mvpmc uses it is
not
possible to grab a single recording's information. It may be
possible if we
were using the xml method that is in the 0.21 and newer myth, and then
it would
be possible to not load/ignore the information that is not often used and
retrieve as necessary.
Here is the current proginfo structure:
[snip large proginfo structure]
So about 200 bytes per program before anything is allocated to any of
the char
*. If something is allocated to the char *'s then there are a
number of bytes
for the reference counters for overhead for each allocation, plus the
bytes
allocated.
I suspect that a number of the entries aren't used anywhere except
show_info and
a number of the entries even if they are used in show_info aren't of
any use to
a person so could probably be ignored.
It's worth noting that a (re-encoded) version of the data in this
structure is passed back to MythTV to request playback of a recording,
so you also need to determin what data in this structure MythTV
uses/needs for file i/o.
I need to figure out a way to gauge how much memory is being used so I
can get
an idea of how much is being saved.
Also, watching recordings through the MythTV interface simply doesn't
work when I use the wireless network - if I start to watch something
it takes a few minutes until anything appears, and then only in short
bursts. I can see on the router lights and statistics that only a few
packets every few seconds are actually transmitted...
However, if I watch the same recording by selecting the .mpg file
through the file browser it plays just fine, and I can see that the
wireless traffic flows perfectly. Could this be because there is much
more free (and less fragmented) memory when playing that way?
My guess would be that this is related to the different block sizes in
use for the MythTV protocol connection compared to the NFS connection,
and them travelling better over NFS. Luckily for you you should be able
to tell mvpmc to use NFS to play the file whilst still browsing through
the MythTV interface. Simply add the -r /path/to/mythtv/mpegs to the
command line options for mvpmc in your dongle.bin.config
Finally, I wanted to ask if anything came from this discussion. Did
anyone manage to test the proposed changes? If not, how long does it
take to get to the point where I can play around with the code myself?
I make my living as a codemonkey but haven't done much private coding
in years except for a few private hacks in mythfilldatabase...
/ Niklas (op)
It should be very easy. The normal build environment automatically
downloads (into ~/downloads) everything required, and then automatically
builds all the toolchains required (into ~/toolchains).
The 2 major pre-requisits are the git version control system, and the
scons "makefile replacement". On a debian system you can get both of
these by doing an:
apt-get install git-core git-gui scons
Whilst the version of git in debian etch does work, it's a little old
and lacks some of the functionality you may want, so if you're a debian
user then I'd recommend getting the version from http://www.backports.org.
Once you have git and scons installed you should be able to simply do:
git clone git://git.mvpmc.org/repos/mvpmc.git
and then:
make
And everything necessary should be downloaded/compiled. I think part of
the toolchain build process still errors when it has succeeded, so you
may find you have to run make a couple of times to get all the way through.
Cheers,
Simon
diff --git a/include/mvp_refmem.h b/include/mvp_refmem.h
index 3bc1d10..fe1a25a 100644
--- a/include/mvp_refmem.h
+++ b/include/mvp_refmem.h
@@ -32,6 +32,9 @@
* -----------------------------------------------------------------
*/
+/* Return current number of references outstanding for everything */
+extern int ref_get_refcount();
+
/**
* Release a reference to allocated memory.
* \param p allocated memory
diff --git a/libs/libcmyth/proglist.c b/libs/libcmyth/proglist.c
index 40d7458..165b444 100644
--- a/libs/libcmyth/proglist.c
+++ b/libs/libcmyth/proglist.c
@@ -52,11 +52,13 @@ static void
cmyth_proglist_destroy(cmyth_proglist_t pl)
{
int i;
-
+ ref_get_refcount("Before cmyth_proglist_destroy");
cmyth_dbg(CMYTH_DBG_DEBUG, "%s\n", __FUNCTION__);
if (!pl) {
return;
}
+
+ printf("Destroying proglist %p with count %ld\n",pl,pl->proglist_count);
for (i = 0; i < pl->proglist_count; ++i) {
if (pl->proglist_list[i]) {
ref_release(pl->proglist_list[i]);
@@ -66,6 +68,7 @@ cmyth_proglist_destroy(cmyth_proglist_t pl)
if (pl->proglist_list) {
free(pl->proglist_list);
}
+ ref_get_refcount("After cmyth_proglist_destroy");
}
/*
@@ -89,6 +92,7 @@ cmyth_proglist_create(void)
cmyth_proglist_t ret;
cmyth_dbg(CMYTH_DBG_DEBUG, "%s\n", __FUNCTION__);
+ ref_get_refcount("before cmyth_proglist_create:");
ret = ref_alloc(sizeof(*ret));
if (!ret) {
return(NULL);
@@ -97,6 +101,8 @@ cmyth_proglist_create(void)
ret->proglist_list = NULL;
ret->proglist_count = 0;
+ ref_get_refcount("after cmyth_proglist_create:");
+
return ret;
}
@@ -317,6 +323,7 @@ cmyth_proglist_get_all_recorded(cmyth_conn_t control)
{
cmyth_proglist_t proglist = cmyth_proglist_create();
+ ref_get_refcount("before cmyth_proglist_get_all_recorded:");
if (proglist == NULL) {
cmyth_dbg(CMYTH_DBG_ERROR,
"%s: cmyth_proglist_create() failed\n",
@@ -333,6 +340,7 @@ cmyth_proglist_get_all_recorded(cmyth_conn_t control)
ref_release(proglist);
return NULL;
}
+ ref_get_refcount("after cmyth_proglist_get_all_recorded:");
return proglist;
}
@@ -359,7 +367,7 @@ cmyth_proglist_t
cmyth_proglist_get_all_pending(cmyth_conn_t control)
{
cmyth_proglist_t proglist = cmyth_proglist_create();
-
+ ref_get_refcount("before cmyth_get_all_pending:");
if (proglist == NULL) {
cmyth_dbg(CMYTH_DBG_ERROR,
"%s: cmyth_proglist_create() failed\n",
@@ -376,6 +384,7 @@ cmyth_proglist_get_all_pending(cmyth_conn_t control)
ref_release(proglist);
return NULL;
}
+ ref_get_refcount("before cmyth_get_all_pending:");
return proglist;
}
diff --git a/libs/librefmem/alloc.c b/libs/librefmem/alloc.c
index ae34d8d..574ec85 100644
--- a/libs/librefmem/alloc.c
+++ b/libs/librefmem/alloc.c
@@ -55,6 +55,8 @@
#define GUARD_MAGIC 0xe3
#endif /* DEBUG */
+static int total_refcount=0;
+static int total_bytecount=0;
/*
* struct refcounter
*
@@ -102,6 +104,12 @@ typedef struct {
static refcounter_t *ref_list[REF_ALLOC_BINS];
#endif /* DEBUG */
+int ref_get_refcount(char *loc)
+ {
+ printf("%s Total Refs: %7d Total Bytes:
%8d\n",loc,total_refcount,total_bytecount);
+ return(total_refcount);
+ }
+
#if defined(DEBUG)
static inline void
ref_remove(refcounter_t *ref)
@@ -228,6 +236,9 @@ __ref_alloc(size_t len, const char *file, const char *func,
int line)
if (block) {
memset(block, 0, sizeof(refcounter_t) + len);
mvp_atomic_set(&ref->refcount, 1);
+ total_refcount ++;
+ total_bytecount += sizeof(refcounter_t) + len;
+
#ifdef DEBUG
ref->magic = ALLOC_MAGIC;
ref->file = file;
@@ -403,6 +414,7 @@ ref_hold(void *p)
assert(guard->magic == GUARD_MAGIC);
#endif /* DEBUG */
mvp_atomic_inc(&ref->refcount);
+ total_refcount ++;
}
refmem_dbg(REF_DBG_DEBUG, "%s(%p) }\n", __FUNCTION__, p);
return p;
@@ -445,6 +457,10 @@ ref_release(void *p)
sizeof(refcounter_t) + ref->length);
assert(guard->magic == GUARD_MAGIC);
#endif /* DEBUG */
+
+/* Remove a refcount */
+ total_refcount --;
+
if (mvp_atomic_dec_and_test(&ref->refcount)) {
/*
* Last reference, destroy the structure (if
@@ -463,6 +479,8 @@ ref_release(void *p)
refmem_ref_remove(ref);
ref->next = NULL;
#endif /* DEBUG */
+/* Remove its bytes */
+ total_bytecount -= ( sizeof(refcounter_t) +
ref->length);
free(block);
}
if (ref->refcount < 0)
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Mvpmc-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mvpmc-users
mvpmc wiki: http://mvpmc.wikispaces.com/
