[dev] Re: refactoring OUString
On 2011/06/08 0:22, Niklas Nebel wrote: Of course we should try to make more use of multiple threads. This isn't a new idea either, see http://wiki.services.openoffice.org/wiki/Calc/Performance/misc. Christian did some experiments with parallel loading a while ago (http://blogs.oracle.com/GullFOSS/entry/xml_performance_and_now_for). The results for Impress weren't spectacular, but Calc or Writer may be different. Yep! I am a multithread, data-driven programming lover, too. :-) On 07.06.2011 13:15, tora - Takamichi Akiyama wrote: 2) Slicing cheese and throwing them out at once For the internal tasks such as Save as and Export to we might get a big advantage. Such a task starts from the framework, calls thousands of methods, and finally leaves the only single value meaning a SUCCESS or FAILURE. No String instance involved during the task is needed to be persistent. On 2011/06/08 0:22, Niklas Nebel wrote: Right now, that isn't entirely true. For example, saving might need to calculate a formula, and the calculated result is then kept in the cell, in a string that continues to be referenced after saving. There might be similar cases elsewhere. These would probably have to be moved into a separate step before saving. Sounds a bit fragile, but then it could actually save a significant amount of time. That is why I would like to encourage programmers to take care of the life time of data. For instance, in the user scenario below, there might be (1) data lasting until the soffice.bin quits. (2) data lasting until a document is closed. (3) data lasting until a current thread ends. (4) data lasting until a certain task finishes. (5) data lasting until a current function call returns. 1. File - New - Spreadsheet 2. work on it and save it. 3. File - Close. In the step 1, construct an instance of memory allocator for (2). In the step 2, use it to allocate memory chunks lasting as long as the document is open. In the step 3, destroy the allocator to completely free the allocated memory. Lessons we might have learned: We can implement and utilize some purpose oriented memory allocators as well as the general, expensive one: malloc() and free(). Programmers may wisely choose what memory allocator is appropriate for questioned data. On the other hand, now might be a perfect time to discuss crazy ideas, without mundane details getting in the way. Aha! here is another crazy ideas :-) https://bitbucket.org/tora/ooo-idea-zstring/src memory_allocator_for_zstring.cxx shows an idea of reusable, cache, memory allocation mechanism for new String class. The key concept here is not to actually free the memory being freed, but to cache it for a later use. Reuse the most recently freed memory first so that the Translation Lookaside Buffer (TLB) achieves higher hit ratio. In contrast, if the oldest freed memory is used first, the entire system performance might suffer because the relevant entry is surely absent from the TLB and, moreover, the relevant memory page might have been swapped out to a disk device. vec.hxx implements a c++ template for cheaply expandable vector. test_vec.cxx demonstrates usage of vec.hxx Best regards, Tora -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: [l10n-dev] How to build OOo with additional localizations
Cross-posting to dev@openoffice.org and d...@l10n.openoffice.org Please follow up to d...@l10n.openoffice.org On 2011/06/09 16:25, Dmitry A. Ashkadov wrote: I have cloned repository DEV300. It doesn't contain localizations. For localizations there is another repository. How can I build OOo with localizations? Should I clone l18n repository to subdirectory l20n inside DEV300 repository? I would like to know about it, too! :-) Here is what I currently have: |-- DEV300_m106 -- http://hg.services.openoffice.org/DEV300 |-- ext_sources `-- l10n -- http://hg.services.openoffice.org/master_l10n/DEV300 |-- Repository.mk |-- RepositoryFixes.mk `-- l10n |-- localization_present.mk |-- prj |-- source |-- unxsoli4 `-- util symbolic link: DEV300_m106/l10n - ../l10n/l10n Best regards, Tora -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: refactoring OUString
On Thu, Jun 9, 2011 at 9:20 AM, tora - Takamichi Akiyama t...@openoffice.org wrote: That is why I would like to encourage programmers to take care of the life time of data. First of, I am doubtful that encouraging manual memory management is a good idea. Errors in manual memory management probably are the cause for the vast majority of severe failures in C/C++ programs. Hence, I would always try to abstract from actual memory as much as possible. (Performance considerations are of course valid, but they must be balanced against safety and maintainability considerations.) What you describe with Slicing cheese and throwing them out at once can be done, but I would not want to do it manually. There are systems more clever than C++, building on effect types and region-based memory management, that exploit such optimizations. But there, it is the language implementation---and not the programmer writing a program in that language---that carries out the proof that keeping data in a region of memory that is discarded wholesale at a certain point in time is sound. That said, it might work to map your various levels of data---from data lasting until the soffice.bin quits to data lasting until a current function call returns---to different C++ types with appropriate conversion functions that potentially need to copy data, to statically ensure sound memory access while on the one hand allowing to exploit optimized memory management strategies and on the other hand still being safe if data does escape from its anticipated level. Would be a nice experiment. -Stephan -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: [l10n-dev] How to build OOo with additional localizations
Hello tora, On Thursday 09 June 2011, 05:31:51, tora - Takamichi Akiyama wrote: Cross-posting to dev@openoffice.org and d...@l10n.openoffice.org Please follow up to d...@l10n.openoffice.org On 2011/06/09 16:25, Dmitry A. Ashkadov wrote: I have cloned repository DEV300. It doesn't contain localizations. For localizations there is another repository. How can I build OOo with localizations? Should I clone l18n repository to subdirectory l20n inside DEV300 repository? I would like to know about it, too! :-) Here is what I currently have: |-- DEV300_m106 -- http://hg.services.openoffice.org/DEV300 |-- ext_sources `-- l10n -- http://hg.services.openoffice.org/master_l10n/DEV300 |-- Repository.mk |-- RepositoryFixes.mk `-- l10n |-- localization_present.mk |-- prj |-- source |-- unxsoli4 `-- util symbolic link: DEV300_m106/l10n - ../l10n/l10n Best regards, Tora I used the following mail to build with localization: http://openoffice.org/projects/www/lists/dev/archive/2011-03/message/4 [I guess this should be copied somewhere in the building guide on the wiki] See also: http://wiki.services.openoffice.org/wiki/Source_config Regards -- Ariel Constenla-Haile La Plata, Argentina signature.asc Description: This is a digitally signed message part.
[dev] Re: [l10n-dev] How to build OOo with additional localizations
Hi Ariel, On 2011/06/09 16:25, Dmitry A. Ashkadov wrote: I have cloned repository DEV300. It doesn't contain localizations. For localizations there is another repository. How can I build OOo with localizations? Should I clone l18n repository to subdirectory l20n inside DEV300 repository? On 2011/06/09 18:13, Ariel Constenla-Haile wrote: I used the following mail to build with localization: http://openoffice.org/projects/www/lists/dev/archive/2011-03/message/4 [I guess this should be copied somewhere in the building guide on the wiki] See also: http://wiki.services.openoffice.org/wiki/Source_config Informative!!! Thanks a lot. Tora smime.p7s Description: S/MIME Cryptographic Signature
[dev] Running OOo on Win7
Hello! Is here somebody who builds OOo on Windows 7? -- Best regards, Dmitry -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: Build DEV300_m106 on Linux breaks in module vcl
On Tue, May 31, 2011 at 2:26 PM, Simon Wilper p9w.vu.31122...@gmx.dewrote: syschild.cxx makes use of jvmaccess::VirtualMachine and ImplTestJavaException in GetParentWindowHandle. But actually it should work --without-java since the code passages in question are surrounded by #ifdef SOLAR_JAVA. Maybe I have a look at this when I find the time for it. Turns out this is caused by changing vcl from the old (solenv/inc based) build system to the new (solenv/gbuild based) one in one of the most recent DEV300_m10X versions. While solenv/inc/*.mk take care to pass -DSOLAR_JAVA to C/C++ compilers only when appropriate, solenv/gbuild/gbuild.mk bluntly always includes it in its gb_GLOBALDEFS. diff --git a/solenv/gbuild/gbuild.mk b/solenv/gbuild/gbuild.mk --- a/solenv/gbuild/gbuild.mk +++ b/solenv/gbuild/gbuild.mk @@ -147,7 +147,6 @@ -DENABLE_LAYOUT_EXPERIMENTAL=0 \ -DENABLE_LAYOUT=0 \ -DOSL_DEBUG_LEVEL=$(gb_DEBUGLEVEL) \ - -DSOLAR_JAVA \ -DSTLPORT_VERSION=$(STLPORT_VER) \ -DSUPD=$(UPD) \ -DVCL \ @@ -179,6 +178,10 @@ endif +ifeq ($(SOLAR_JAVA),TRUE) +gb_GLOBALDEFS += -DSOLAR_JAVA +endif + ifneq ($(strip $(ENABLE_GTK)),) gb_GLOBALDEFS += -DENABLE_GTK endif should fix that. -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: Running OOo on Win7
On 09.06.2011 11:29, Dmitry A. Ashkadov wrote: Hello! Is here somebody who builds OOo on Windows 7? Yes, using Windows 7 and cygwin 1.7.7 Regards, Mathias -- Mathias Bauer (mba) - Project Lead OpenOffice.org Writer OpenOffice.org Engineering at Oracle: http://blogs.sun.com/GullFOSS Please don't reply to nospamfor...@gmx.de. I use it for the OOo lists and only rarely read other mails sent to it. -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: Running OOo on Win7
Hi Dmitry, yes, I successfully built on both code lines and the application worked fine. Admittedly builds on OOO340 and the latest milestones of DEV300 needed some patches, but not to run the built office, without the patched the build itself broke. Regards, Mathias On 09.06.2011 12:07, Dmitry A. Ashkadov wrote: Does the office start successful? My OOo build crashes at startup (DEV300 and OOO340) without any information about problem. Cygwin console is empty. 09.06.2011 13:44, Mathias Bauer пишет: On 09.06.2011 11:29, Dmitry A. Ashkadov wrote: Hello! Is here somebody who builds OOo on Windows 7? Yes, using Windows 7 and cygwin 1.7.7 Regards, Mathias -- Mathias Bauer (mba) - Project Lead OpenOffice.org Writer OpenOffice.org Engineering at Oracle: http://blogs.sun.com/GullFOSS Please don't reply to nospamfor...@gmx.de. I use it for the OOo lists and only rarely read other mails sent to it. -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: Running OOo on Win7
I tried extract archive to home folder to reduce length of path, and the start office. It doesn't crash at startup now. So, the problem is a long paths. 09.06.2011 17:35, Mathias Bauer пишет: Hi Dmitry, yes, I successfully built on both code lines and the application worked fine. Admittedly builds on OOO340 and the latest milestones of DEV300 needed some patches, but not to run the built office, without the patched the build itself broke. Regards, Mathias On 09.06.2011 12:07, Dmitry A. Ashkadov wrote: Does the office start successful? My OOo build crashes at startup (DEV300 and OOO340) without any information about problem. Cygwin console is empty. 09.06.2011 13:44, Mathias Bauer пишет: On 09.06.2011 11:29, Dmitry A. Ashkadov wrote: Hello! Is here somebody who builds OOo on Windows 7? Yes, using Windows 7 and cygwin 1.7.7 Regards, Mathias -- Best regards, Dmitry -- - To unsubscribe send email to dev-unsubscr...@openoffice.org For additional commands send email to sy...@openoffice.org with Subject: help
[dev] Re: refactoring OUString
Sorry, this mail is too long...
On Thu, Jun 9, 2011 at 9:20 AM, tora - Takamichi Akiyama t...@openoffice.org
mailto:t...@openoffice.org wrote:
That is why I would like to encourage programmers to take care of the life
time of data.
I know that that statement is controversial.
On 2011/06/09 18:02, Stephan Bergmann wrote:
First of, I am doubtful that encouraging manual memory management is a good
idea. Errors in manual memory management probably are the cause for the vast
majority of severe failures in C/C++ programs.
Please be noticed that I don't say programmers should need to explicitly call
memory management related functions such as malloc() or free().
Rather, I would like to suggest thinking of the characteristics of the
questioned data.
1. Delegation of the responsibility to choose a type of memory allocator
To achieve both stability and performance at the same time, I would like to propose Don't do
all of it in the SAL, rather Delegate certain responsibility to its users, i.e.
programmers.
Who knows the type of life time of data? SAL does? No. The programmers do.
Life time of data
(1) data lasting until the soffice.bin quits.
(2) data lasting until a document is closed.
(3) data lasting until a current thread ends.
(4) data lasting until a certain task finishes.
(5) data lasting until a current function call returns.
(6) data lasting until a current block ends.
Multithread awareness
(a) data that is shared with more than one threads.
(b) data that is used in the only this thread.
Asynchronous awareness
(i) data that is used in a asynchronously called function such as a signal
handler.
(ii) data that is used in a normal function.
2. Potential dead lock
A code for crash reporter has a potential, dead lock problem.
http://hg.services.openoffice.org/DEV300/file/tip/sal/osl/unx/backtrace.c
Asynchronous-unsafe functions such as fprintf() are used in the context of
signal handler.
Consider this situation:
1. A Segment violation, aka SEGV occurs in malloc() or free() due to memory
corruption. Such a function holds the global mutex lock.
2. The first call of fprintf(), it internally calls malloc() to obtain a memory
area as a text buffer. Then a dead lock occurs.
For that topic, I would be posing a question later.
Hence, I would always try to abstract from actual memory as much as possible.
(Performance considerations are of course valid, but they must be balanced
against safety and maintainability considerations.)
3. Come up with the exciting measures
There in no need to keep relying on the traditional approaches invented in the
20th century.
With my experiences from 8 bit processor, I certainly believe the programmers'
awareness of how memory area is treated is the crucial factor to achieve
performance, safety, and maintainability at the same time.
I do not have an objection against your idea abstraction, though.
=
// Slicing cheese and throwing them out at once
#define ALLOCATION_SIZE ( 1024 * 1024 ) // 1MB
#define ALIGNMENT 4
void* SCATTOAO::xmalloc( size_t nSize )
{
nSize = ( ( nSize - 1 ) / ALIGNMENT + 1 ) * ALIGNMENT;
if ( m_nRest nSize ) {
nAllocationSize = ( ( nSize - 1 ) / ALLOCATION_SIZE + 1 ) *
ALLOCATION_SIZE;
p = memory_page_allocation( nAllocationSize, PRIVATE|ANONIMOUS );
m_vector.append( Entry( p, nAllocationSize ) );
m_pNose = p;
n_nRest = nAllocationSize;
}
ret = m_pNose;
m_pNose += nSize; // Slice a block of cheese
m_nRest -= nSize;
return (void *) ret;
}
void SCATTOAO::xfree( void* )
{
// do nothing at all
}
SCATTOAO::~SCATTOAO()
{
if ( Applicatoin::IsMemoryCheckRequested() )
for ( iterator m_vector ) // Turn them to be a trap
alter_page_attribute( *it, NO_READ_ACCESS|NO_WRITE_ACCESS|NO_EXEC );
else
for ( iterator m_vector ) // Throw them at once
memory_page_deallocation( it-m_pAddress, it-m_nSize );
}
=
Please have a look at an additional code fragment in the destructor above:
if ( Applicatoin::IsMemoryCheckRequested() )
for ( iterator m_vector ) // Turn them to be a trap
alter_page_attribute( *it, NO_READ_ACCESS|NO_WRITE_ACCESS|NO_EXEC );
1. soffice.bin is invoked with a new command line option such as -memorycheck
2. Applicatoin::IsMemoryCheckRequested() returns TRUE.
3. The memory pages being freed turns to be a trap.
4. A problematic code mistakenly attempts to read or write data in the
already-freed-memory-area.
5. The trap sets off the alarm and an interruption is sent by the OS.
6. A signal handler in the SAL catches the interruption.
7. A crash report that reveals the exact location of the code is made.
We have been cultivating thousands of test scenarios for more than a decade.
Just leave the qatesttool running for a day and night with the option
-memorycheck.
4. Utilizing the cutting-edge technology invented in the 21th century.
solaris$ cat
