Re: [Harbour] Debugging c code
On Sun, 07 Feb 2010, April White wrote: Hi, Przemek, I want to confirm something. I'm not yet using the new code you wrote, so bear with me here. If I allocate memory using hb_xgrab_fm() as you suggested before and then call p1 = hb_itemNew(p2) if I do not release p1 the line reported by FM statistics appears to be where hb_xgrab_fm() is called. This is what I believe I am seeing, can you confirm this? It depends on few different things. By default when HB_TR_LEVEL is not set to HB_TR_DEBUG hb_traceset( HB_TR_FM, szFile, iLine, NULL ); set's the information about file and line used by the nearest call to hb_xgrab()/hb_xalloc() and after using it it's reset so it can be used only once. In such case if you make: hb_traceset( HB_TR_FM, MYFILE, 123, NULL ); ptr = hb_xgrab( 100 ); // use MYFILE:123 set above pItem = hb_itemNew( NULL ); // use PROCNAME():PROCLINE() but if you make: hb_traceset( HB_TR_FM, MYFILE, 123, NULL ); ptr = hb_xgrab( 100 ); // use MYFILE:123 hb_traceset( HB_TR_FM, MYFILE2, 125, NULL ); pItem = hb_itemNew( NULL ); // use MYFILE2:125 then 1-st memory block allocated by hb_xgrab()/hb_xalloc() inside hb_itemNew() will use MYFILE2:125 and next ones (if any) will use PROCNAME():PROCLINE(). If you set HB_TR_LEVEL to HB_TR_DEBUG then hb_traceset() will set default file and line number for all blocks until next call to hb_traceset() or HB_TRACE() message in C code because compiled into final code HB_TRACE() messages (not stripped but C macrocompiler) overwrite values set by hb_traceset(). So you described situation which happens when core code is compiled without HB_TRACE() debug messages (default) and you set HB_TR_LEVEL envvar to HB_TR_DEBUG. BTW you can define macro HB_FM_MARKER and use it in your code before functions which allocates memory: #ifdef DEBUG #define HB_FM_MARKER hb_traceset( HB_TR_FM, __FILE__, __LINE__, NULL ); #else #define HB_FM_MARKER #endif [...] HB_FM_MARKER ptr = hb_xgrab( 100 ); HB_FM_MARKER pItem = hb_itemNew( NULL ); best regards, Przemek ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
Przemek, I want to confirm something. I'm not yet using the new code you wrote, so bear with me here. If I allocate memory using hb_xgrab_fm() as you suggested before and then call p1 = hb_itemNew(p2) if I do not release p1 the line reported by FM statistics appears to be where hb_xgrab_fm() is called. This is what I believe I am seeing, can you confirm this? April -- With my last breath I spit at thee... 'The Wrath of Khan' ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
On 2010 Feb 3, at 03:56, Xavi wrote: To Viktor, El 03/02/2010 0:09, Viktor Szakáts escribió: for GPF at application startup caused by accessing TSD before HVM stack is initialized. It seems to work though it can be greatly simplified and of course the GPF in startup code and non HVM threads has to be fixed. That pretty much sums up what I didn't like about the implementation without even knowing the exact details (I'd have bet that MT mode had problems f.e.) That's great!. No Comments. :-) Next time maybe you should post your patch instead of hiding it, so no guessing is required based on past work! Brgds, Viktor ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
To get the support (or to report bug) about your Ubuntu issue please refer to the main https://launchpad.net/ubuntu Ubuntu site, be free to ask any kind of question here: https://answers.launchpad.net/ubuntu Hth ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
On Mon, 01 Feb 2010, Xavi wrote:
Hi,
HB_TRACEFM(), with my next commit. I'm waiting for Przemek and others
are ready. ;)
Can you send .diff file here so we can check what exactly this
modification does?
Is there a macro I can use to identify this sub-function? I could leave
it in the code wrapped conditional code for debug only.
Yes if you have a C99 compiler, just have to redefine the macro in your code
.-
#ifdef HB_TRACEFM
# undef HB_TRACEFM
# define HB_TRACEFM( x ) hb_fm_xInfoEx( #x, __FILE__, __LINE__, __func__ );
x
#endif
We do not have hb_fm_xInfoEx() in our code so I guess it's part of
modifications you want to commit. __func__ is C99 macro. It makes
log more readable but __FILE__ + __LINE__ is enough to detect exact
place and function name if you have source code so it's not critical
information. For current unmodified code requested functionality
(without function name) can be reached by:
extern HB_EXPORT void * hb_xgrab_fm( ULONG ulSize,
const char * szFile, int iLine );
[...]
void * hb_xgrab_fm( ULONG ulSize, const char * szFile, int iLine )
{
hb_tr_file_ = szFile;
hb_tr_line_ = iLine;
return hb_xgrab( szFile );
}
#if HB_TR_LEVEL = HB_TR_DEBUG
#define HB_XGRABFM( n )hb_xgrab_fm( n, __FILE__, __SIZE__ )
#else
#define HB_XGRABFM( n )hb_xgrab( n )
#endif
with most of C compilers the same effect can be reached using only
macros without hb_xgrab_fm() function, i.e.:
#if HB_TR_LEVEL = HB_TR_DEBUG
#define HB_XGRABFM( n )( hb_tr_file_ = __FILE__, \
hb_tr_line_ = __LINE__, \
hb_xgrab( n ) )
#else
#define HB_XGRABFM( n )hb_xgrab( n )
#endif
If user wants does not want to set HB_TR_LEVEL=DEBUG then the
above macro can be extend to dynamically change and restore
debug level, i.e.:
#if HB_TR_LEVEL = HB_TR_DEBUG
#define HB_XGRABFM( n ){ int iLevel = hb_tracelevel( HB_TR_DEBUG ); \
void * ptr; \
hb_tr_file_ = __FILE__; \
hb_tr_line_ = __LINE__; \
ptr = hb_xgrab( n ); \
hb_tracelevel( iLevel ); \
( void ) ptr; }
#else
#define HB_XGRABFM( n )hb_xgrab( n )
#endif
anyhow such version can be compiled only by few C compilers, i.e. by GCC.
To make it simpler we can change a little bit fm.c and HB_TRACE_STEALTH()
macro so user can use the first version independently to HB_TR_LEVEL setting.
BTW such modifications in HB_TRACE_STEALTH() macro should be done also to
fix yet another problem in current code so I'll commit them ASAP.
best regards,
Przemek
___
Harbour mailing list (attachment size limit: 40KB)
Harbour@harbour-project.org
http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
On Tue, 02 Feb 2010, Xavi wrote:
Hi,
Thank you Przemek. It's not so easy because these solutions don't
work with C++ new operator.
If you want to include memory allocated by C++ code to FM static
then it's necessary to overload new and delete operators and redirect
them to hb_xgrab()/hb_xfree(). hbmk2 has an option to enable such
overloading. Anyhow C preprocessor is too week to automatically insert
instruction which can safe __FILE__ and __LINE__ values so it cannot
be cleanly done. Using macros like HB_TRACEFM( x ) you send is not
solution too because in C++ code NEW/DELETE operators can be executed
indirectly. I.e.:
{
my_type x;
f( x );
}
can activate code constructor and destructors which executes indirectly
some other code which can allocated new memory. To correctly set file
and line information in such code you will have to make sth like:
{
HB_TRACEFM( my_type x ); // this code is illegal in ANSI C mode !!!
HB_TRACEFM( f( x ) );
HB_TRACEFM( ) // dummy call to set valid line and file info for
// destructors which also can allocate some memory
}
In fact for real C++ code using extensively constructors and destructors
you will have to enclose nearly _each_ line of code inside HB_TRACEFM()
so for me HB_TRACEFM() isn't reasonable solution, too.
Please also note that:
HB_TRACEFM( my_type x );
just like:
HB_TRACEFM( void * ptr = hb_xgrab( 100 ) );
generates in hidden way code which is not ANSI C compatible what creates
yet another problem.
Instead of using such HB_TRACEFM() macro I suggest to simply define
macro which will set file and line number for FM module, i.e.:
#define HB_TR_SET_CREATE(l) do { \
hb_tr_level_ = l; \
hb_tr_file_ = __FILE__; \
hb_tr_line_ = __LINE__; \
} while( 0 )
#if HB_TR_LEVEL = HB_TR_DEBUG
#define HB_TR_SET_HB_TR_DEBUG() HB_TR_SET_CREATE(HB_TR_DEBUG)
#else
#define HB_TR_SET_HB_TR_DEBUG()
#endif
[...]
#if 1 /* always! */
#define HB_TR_SET_HB_TR_ALWAYS()HB_TR_SET_CREATE(HB_TR_ALWAYS)
#else
#define HB_TR_SET_HB_TR_ALWAYS()
#endif
#define HB_TR_SET(l) HB_TR_SET_##l()
so user can mark any place in his code before calling hb_xgrab()
or activating any other code which may allocates new memory by simple
adding HB_TR_SET( HB_TR_DEBUG ); i.e.
HB_TR_SET( HB_TR_DEBUG ); // code marker, store file and line info
ptr = hb_xgrab( 100 );
In such version it's explicitly visible for users what such macros does
and which lines will be reported by FM statistic module.
And if you still prefer HB_TRACEFM() then you can simply define in your
code:
HB_TRACEFM( x ) HB_TR_SET( HB_TR_ALWAYS ); x
In the code you created you are storing with each memory block also
the code inside HB_TRACEFM() macro ( #x ) in the 264 bytes buffer
attached to each memory block. For me it seems to be redundant. Having
file name and line number is enough to check the exact code so it does
not give any new functionality but increase the overhead created by
FM statistic module reducing the speed and increasing total memory
usage. I agree that final FM log looks nicer with such feature but
personally I do not think that such reason is important enough. Anyhow
this are only my personal feelings so it should be rather group
decision. If other developers things that it's important to attach
text with source code inside HB_TRACEFM() macro to each memory block
allocated by FM statistic module to report such lines in fm.log file
and agree to accept slowness and additional memory overhead because
information about source file name and line number is not enough
then we can add it.
Attache files ready for committed, I've taken care to preserve intact
the largest existing code as possible.
Please, you can review especially in MT mode. I've tried it with
success in a large multi-threaded application without conflict or
deadlock but maybe I missed something and I don't know force the
error.
I see that you used TSD structure allocated on HVM stack to make it
MT safe. But it creates one serious problem. It means that it's
not possible to activate this code before HVM stack is created
by application startup code and it's not possible to use hb_xgrab()
by non HVM threads. I only guess that this trick with s_pInfoEx
which makes it much more complicated then necessary is workaround
for GPF at application startup caused by accessing TSD before HVM
stack is initialized. It seems to work though it can be greatly
simplified and of course the GPF in startup code and non HVM threads
has to be fixed.
In general whole HB_TRACE() module should be fixed to be MT safe.
Unfortunately it can be cleanly done without any TSD/TLS variables
only using C99 macros :-(.
Anyhow it systematically begins to be bigger problems so I'll
Re: [Harbour] Debugging c code
To Xavi and All,
Thank you Przemek. It's not so easy because these solutions don't
work with C++ new operator.
If you want to include memory allocated by C++ code to FM static
then it's necessary to overload new and delete operators and redirect
them to hb_xgrab()/hb_xfree(). hbmk2 has an option to enable such
overloading. Anyhow C preprocessor is too week to automatically insert
instruction which can safe __FILE__ and __LINE__ values so it cannot
be cleanly done. Using macros like HB_TRACEFM( x ) you send is not
solution too because in C++ code NEW/DELETE operators can be executed
indirectly. I.e.:
{
my_type x;
f( x );
}
can activate code constructor and destructors which executes indirectly
some other code which can allocated new memory. To correctly set file
and line information in such code you will have to make sth like:
{
HB_TRACEFM( my_type x ); // this code is illegal in ANSI C mode !!!
HB_TRACEFM( f( x ) );
HB_TRACEFM( ) // dummy call to set valid line and file info for
// destructors which also can allocate some memory
}
In fact for real C++ code using extensively constructors and destructors
you will have to enclose nearly _each_ line of code inside HB_TRACEFM()
so for me HB_TRACEFM() isn't reasonable solution, too.
Please also note that:
HB_TRACEFM( my_type x );
just like:
HB_TRACEFM( void * ptr = hb_xgrab( 100 ) );
...
generates in hidden way code which is not ANSI C compatible what creates
yet another problem.
Instead of using such HB_TRACEFM() macro I suggest to simply define
macro which will set file and line number for FM module, i.e.:
#define HB_TR_SET_CREATE(l) do { \
hb_tr_level_ = l; \
hb_tr_file_ = __FILE__; \
hb_tr_line_ = __LINE__; \
} while( 0 )
#if HB_TR_LEVEL = HB_TR_DEBUG
#define HB_TR_SET_HB_TR_DEBUG() HB_TR_SET_CREATE(HB_TR_DEBUG)
#else
#define HB_TR_SET_HB_TR_DEBUG()
#endif
[...]
#if 1 /* always! */
#define HB_TR_SET_HB_TR_ALWAYS()HB_TR_SET_CREATE(HB_TR_ALWAYS)
#else
#define HB_TR_SET_HB_TR_ALWAYS()
#endif
#define HB_TR_SET(l) HB_TR_SET_##l()
so user can mark any place in his code before calling hb_xgrab()
or activating any other code which may allocates new memory by simple
adding HB_TR_SET( HB_TR_DEBUG ); i.e.
HB_TR_SET( HB_TR_DEBUG ); // code marker, store file and line info
ptr = hb_xgrab( 100 );
In such version it's explicitly visible for users what such macros does
and which lines will be reported by FM statistic module.
And if you still prefer HB_TRACEFM() then you can simply define in your
code:
HB_TRACEFM( x ) HB_TR_SET( HB_TR_ALWAYS ); x
In the code you created you are storing with each memory block also
the code inside HB_TRACEFM() macro ( #x ) in the 264 bytes buffer
attached to each memory block. For me it seems to be redundant. Having
file name and line number is enough to check the exact code so it does
not give any new functionality but increase the overhead created by
FM statistic module reducing the speed and increasing total memory
usage. I agree that final FM log looks nicer with such feature but
personally I do not think that such reason is important enough. Anyhow
this are only my personal feelings so it should be rather group
decision. If other developers things that it's important to attach
text with source code inside HB_TRACEFM() macro to each memory block
allocated by FM statistic module to report such lines in fm.log file
and agree to accept slowness and additional memory overhead because
information about source file name and line number is not enough
then we can add it.
Attache files ready for committed, I've taken care to preserve intact
the largest existing code as possible.
Please, you can review especially in MT mode. I've tried it with
success in a large multi-threaded application without conflict or
deadlock but maybe I missed something and I don't know force the
error.
I see that you used TSD structure allocated on HVM stack to make it
MT safe. But it creates one serious problem. It means that it's
not possible to activate this code before HVM stack is created
by application startup code and it's not possible to use hb_xgrab()
by non HVM threads. I only guess that this trick with s_pInfoEx
which makes it much more complicated then necessary is workaround
for GPF at application startup caused by accessing TSD before HVM
stack is initialized. It seems to work though it can be greatly
simplified and of course the GPF in startup code and non HVM threads
has to be fixed.
That pretty much sums up what I didn't like about the
implementation without even knowing the exact details
(I'd have bet that MT mode had problems f.e.)
And while I liked
Re: [Harbour] Debugging c code
Thank you Przemek. I've tried your suggestion but my implementation
does not work as expected, the stats output is:
fm.c:1134: HB_TR_ERROR Block 1 (size 40) HB_BTREEINSERT(0),
F8A454004001550058D
C4B00010001001000F03F0A000200
But I know the memory allocation is in a sub-function.
My implementation is different from yours but keeps the spirit. I added
some debug code not shown below so I know it is compiling this code and
executing it. My system is:
WinXP SP2
MinGW32 from Qt installer
HB_USER_CFLAGS to compile harbour:
-O3 -Wall -Wunused-variable -Wpointer-arith -DHB_FM_STATISTICS
HB_USER_CFLAGS to compile this project:
-O3 -Wall -Wunused-variable -Wpointer-arith -DHB_FM_STATISTICS
-DHB_TR_LEVEL=5
My existing code already uses a local function to memory allocation, a
wrapper for hb_xgrab() or hb_xrealloc().
#if HB_TR_LEVEL = HB_TR_DEBUG
#define BufferAlloc(n) hb_xgrab_fm( NULL, (n), __FILE__, __LINE__ )
#define BufferRealloc(p,n) hb_xgrab_fm( (p), (n), __FILE__, __LINE__ )
extern HB_EXPORT void * hb_xgrab_fm( void *buffer, ULONG ulSize, const
char * szFile, int iLine )
{
void *tmpBuffer;
hb_tr_file_ = szFile;
hb_tr_line_ = iLine;
tmpBuffer = buffer ? hb_xrealloc( buffer, ulSize ) : hb_xgrab( ulSize );
return tmpBuffer;
}
#else
...
#endif
--
Without facts, the decision cannot be made logically. You must rely on your
human intuition.
- Spock, 'Assignment: Earth', stardate unknown
___
Harbour mailing list (attachment size limit: 40KB)
Harbour@harbour-project.org
http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
On Tue, 02 Feb 2010, April White wrote:
Hi,
Thank you Przemek. I've tried your suggestion but my implementation
does not work as expected, the stats output is:
fm.c:1134: HB_TR_ERROR Block 1 (size 40) HB_BTREEINSERT(0),
F8A454004001550058D
C4B00010001001000F03F0A000200
But I know the memory allocation is in a sub-function.
You haven't set DEBUG tracing level. You can do that setting
HB_TR_LEVEL envvar, i.e.:
export HB_TR_LEVEL=HB_TR_DEBUG
or in DOS/Windows:
set HB_TR_LEVEL=HB_TR_DEBUG
Just try.
I'll commit in a while modifications which allow to eliminate
setting HB_TR_LEVEL using HB_TR_FM message level.
You will have to modify your code to:
void * hb_xgrab_fm( void *buffer, ULONG ulSize,
const char * szFile, int iLine )
{
hb_traceset( HB_TR_FM, szFile, iLine, NULL );
return hb_xrealloc( buffer, ulSize );
}
You can use hb_xrealloc( NULL, ulSize ) to allocate memory so you do not
need additional condition in hb_xgrab_fm() function.
best regards,
Przemek
___
Harbour mailing list (attachment size limit: 40KB)
Harbour@harbour-project.org
http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
To Viktor, El 03/02/2010 0:09, Viktor Szakáts escribió: for GPF at application startup caused by accessing TSD before HVM stack is initialized. It seems to work though it can be greatly simplified and of course the GPF in startup code and non HVM threads has to be fixed. That pretty much sums up what I didn't like about the implementation without even knowing the exact details (I'd have bet that MT mode had problems f.e.) That's great!. No Comments. :-) Best regards, Xavi ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
Hi April, HB_TRACEFM(), with my next commit. I'm waiting for Przemek and others are ready. ;) Is there a macro I can use to identify this sub-function? I could leave it in the code wrapped conditional code for debug only. Yes if you have a C99 compiler, just have to redefine the macro in your code .- #ifdef HB_TRACEFM # undef HB_TRACEFM # define HB_TRACEFM( x ) hb_fm_xInfoEx( #x, __FILE__, __LINE__, __func__ ); x #endif Best regards, Xavi El 01/02/2010 4:00, April White escribió: When my C level code does not release memory, the fm statistics reports this but gives the public function though in some cases the allocation occured in a sub-function. Is there a macro I can use to identify this sub-function? I could leave it in the code wrapped conditional code for debug only. April ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
Re: [Harbour] Debugging c code
Hi, HB_TRACEFM(), with my next commit. I'm waiting for Przemek and others are ready. ;) Is there a macro I can use to identify this sub-function? I could leave it in the code wrapped conditional code for debug only. Yes if you have a C99 compiler, just have to redefine the macro in your code .- #ifdef HB_TRACEFM # undef HB_TRACEFM # define HB_TRACEFM( x ) hb_fm_xInfoEx( #x, __FILE__, __LINE__, __func__ ); x #endif Sorry, I still don't like it in its current form, pls could post the complete patch for review if possible. Brgds, Viktor ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
RE: [Harbour] Debugging c code
You may recall my email from about a week ago about my Ubuntu machine. I'm (re)installed 9.10 with the same results that it does not boot. I do not know enough about linux / ubuntu to know whether it is a bug or a hardware failure. Because the installation has no failures it says to me it is a bug not hardware. Thank you! April p.s. I've found a work around for the runtime error in the btree code, but I do not know 'why' is it happening. More work to do. Subject: Re: [Harbour] Debugging c code From: harbour...@syenar.hu Date: Mon, 1 Feb 2010 16:10:00 +0100 To: harbour@harbour-project.org Hi April, Under Ubuntu you can use valgrind to find out the exact locations of leaked allocations. Check the debugging section in INSTALL. On 2010 Feb 1, at 04:00, April White wrote: When my C level code does not release memory, the fm statistics reports this but gives the public function though in some cases the allocation occured in a sub-function. Is there a macro I can use to identify this sub-function? I could leave it in the code wrapped conditional code for debug only. April ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
[Harbour] Debugging c code
When my C level code does not release memory, the fm statistics reports this but gives the public function though in some cases the allocation occured in a sub-function. Is there a macro I can use to identify this sub-function? I could leave it in the code wrapped conditional code for debug only. April ___ Harbour mailing list (attachment size limit: 40KB) Harbour@harbour-project.org http://lists.harbour-project.org/mailman/listinfo/harbour
