Hi Bart,
thank you for the answer.
Yes perfect, you understood well the problem and your solution is a good
compromise! ;)
Thierry.
From: bartlomie...@zend.com
To: pdt-dev@eclipse.org
Date: Fri, 13 May 2016 13:05:04 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Hi Thierry,
In general canceling indexing operation doesn't work for both indexers (current
implementation for DLTK indexer doesn't care too much about cancel - see
AbstractJob#isCancelled and ProgressJob). So while indexing source modules from
huge project you can
try to cancel it but it won't work as expected. The difference is only while
closing IDE - as indexing thread is daemon, when you close IDE while being in
the middle of indexing then in case of H2 everything will be just dropped when
thread dies. In case of
Lucene we wait for all indexing jobs to be finished as we must close index
writers gracefully.
Anyway we can fix it of course :-). In general i plan to add patch proposal to
DLTK that will enable canceling. It will give a possibility to cancel indexing
in "Progress" view as well as while closing IDE ("Saving workspace state..."
window where you can
cancel ongoing operations). Anyway there might be some side-effect when
canceling in "Progress" view as index might not be complete in that case and it
will be rebuild/updated after next clean or IDE restart (but I guess that user
should be aware of it while
pressing "Stop" :-).
What do you think about it?
Greetings,
Bart
On 5/12/2016 10:32 PM, thierry blind wrote:
Hi Bart,
I have another question concerning the lucene indexer:
I was wondering why it's not possible to interrupt an indexing operation (for
example to close eclipse asap and restart the indexing operation later).
With the H2 indexer I could stop the indexing (after some clicks on the stop
button in the progress panel), or even close eclipse and then hit the indexing
stop button.
Now you have no more control on the indexing process and the progress bar (with
its stop button) becomes totally useless, and even worse, when you close
eclipse, it continues the indexing as a background process, taking a lot of
memory (and time)! Or you can
kill eclipse...
When you take the big projects I've sent you, indexing takes time and you are
"forced" to wait that indexing ends before being able to close eclipse, which
is not the case with classic java projects and their build process. I see that
as a regression, but maybe
I'm too rude :S
Thierry.
From: thierrybl...@msn.com
To: pdt-dev@eclipse.org
Date: Wed, 4 May 2016 16:23:07 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Thank you ! I'm glad to help ;) And thank you for your patience ;)
Thierry.
From: bartlomie...@zend.com
To: pdt-dev@eclipse.org
Date: Wed, 4 May 2016 16:03:01 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Thierry, your help is invaluable :) You pointed me into the right direction. I
didn't check the scenario with dependent projects but it looks like it is the
root cause of the problems.
Thanks again for your help and have a nice holiday leave! :)
On 5/4/2016 5:53 PM, thierry blind wrote:
Maybe in my case I simply though it does loop forever, because I had a
looooooot of big projects depending on same big project "A" (by doing
Properties -> PHP -> Include Path -> Projects -> Add -> projet A).
And it took forever with a big amount of place taken by
org.eclipse.dltk.core.index.lucene
Strangely it was much faster on a new and clean workspace :S
If I have for example 5 projects that depends on a common project "A", the
project "A" will then be indexed 5 times.
In my case with H2 indexer it took 1.1 GB, with the Lucene indexer it takes
more than 3GB on a safe and new workspace!
So really sorry for pointing in the wrong direction :/
And yes there is still room for improvements ;)
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 15:39:01 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
OK, great. For now I have found something else while using your workspace that
is not OK. Even if indexing finishes OK there are some source modules being
re-indexed while closing/restarting IDE. At least it might give me some clues
what might happened as I
think that the problem is related somehow to storing/reading source files
timestamps as this is the place where some redundant indexing might be
triggered.
On 5/4/2016 5:07 PM, thierry blind wrote:
I'll repackage another workspace and tell you when it's available ok ? Maybe I
removed too much...
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 15:00:53 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
I used your workspace in my dev env but unfortunately indexer finished without
a problem. It only took some time to close the PDT running threads (after
closing IDE) as Lucene was merging and closing huge indexes (for about 45-60
seconds) but it finished it
gracefully. Anyway I will try to do my best to reproduce it ;)
On 5/4/2016 4:13 PM, thierry blind wrote:
With pleasure ;)
You can send it at my email address:
thierrybl...@msn.com
Kaloyan, if you're also interested, tell me ok?
I really hope you will also reproduce this bug on your eclipse, I will go crazy
if I'm the only one to have it! lol
Of course each time you start your eclipse you have to delete
.metadata/org.eclipse.dltk.core.index.lucene
And some additional infos on what's installed in my eclipse Mars.2 Release:
Eclipse IDE for Java EE Developers 4.5.2.20160218-0600 epp.package.jee
null
JavaHL 1.7.9 Win32 Binaries (Optional) 5.0.0.I20150529-1700
org.polarion.eclipse.team.svn.connector.javahl17.win32.feature.group
Polarion Software
PDT Extensions 0.22.2.201604260159
org.pdtextensions.feature.feature.group
http://pdt-extensions.org
PHP CodeSniffer 1.3.0.R20111119000000
org.phpsrc.eclipse.pti.tools.codesniffer.feature.group Sven Kiera
PHP Development Tools (PDT) 4.0.0.201605031251
org.eclipse.php.feature.group Eclipse PDT
PHP Development Tools (PDT) Mylyn Integration 4.0.0.201605031251
org.eclipse.php.mylyn.feature.group Eclipse PDT
PHP Development Tools (PDT) Source Code 4.0.0.201605031251
org.eclipse.php.source.feature.group Eclipse PDT
PHP Tool Integration Core 1.6.0.R20111119000000
org.phpsrc.eclipse.pti.core.feature.group Sven Kiera
PHPUnit 0.8.0.R20111119000000
org.phpsrc.eclipse.pti.tools.phpunit.feature.group Sven Kiera
PMD Plug-in 4.0.8.v20151204-2156
net.sourceforge.pmd.eclipse.feature.group PMD Project
Subversive SVN Team Provider 3.0.4.I20160131-1700
org.eclipse.team.svn.feature.group Eclipse.org
Thierry.
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 14:02:11 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
That would be great! Thank you one more time for all the feedback! This zipped
workspace would be very appreciated, I will send you soon my private address
that you can send the google drive link at.
On 5/4/2016 3:50 PM, thierry blind wrote:
It won't be possible at work for Skype sorry :S
And I followed progress bar since first try ;)
Here is what happen : I have PHP projects A, B, C and D. Projects B, C and D
depend on project A.
So indexer starts with project A, continues with project B (only few files),
goes back to A, does a little bit from C, goes back to A, does a little bit
from D, and then
it seems it only loops inside of A. It seems that if I only have project A
open in my workspace, indexer works correctly and then I can open project B and
indexer still works correctly, and so on...
I'll try to exclude most of the projects of my workspace , zip my workspace and
share it on my google drive, would it be ok?
The workspace will look awful with lot of broken things, but mainly it will
continue to loop forever (hopefully also for you!), it's the most important no
? ;)
Of course there will still be private stuff inside, please give me a private
email address, so I can send you the google drive link ;)
Thierry.
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 11:59:33 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Unfortunately nothing special in the thread dumps. Could you possibly take a
look at the indexing progress bar in "Progress" view while having this infinite
indexing? Does the names of the files being indexed change or progress stuck
only on a one particular
file? Maybe it would be possible for you to arrange some short Skype call so
you can show me what is going on?
On 5/4/2016 1:38 PM, thierry blind wrote:
Here you have :)
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 09:39:25 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Thierry, could you possibly make some thread dump in a time when there is this
infinite looping? I think that it might tell me something more.
On 5/4/2016 11:32 AM, thierry blind wrote:
Yes ;)
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 09:29:49 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Did you delete "org.eclipse.dltk.core.index.lucene" from .metadata before
second run?
On 5/4/2016 11:15 AM, thierry blind wrote:
Second run, same loop problem, I had to kill eclipse, all errors that you see
in log files happen when closing eclipse.
I'll create a fresh new workspace, copy a set of my PHP projects, import them
and see what indexer does...
Thierry.
From:
thierrybl...@msn.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 08:29:03 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
I'll give a try and tell you ;)
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Wed, 4 May 2016 08:24:21 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Is it still the same after closing PDT, deleting
"org.eclipse.dltk.core.index.lucene" and starting IDE once again?
On 5/4/2016 10:17 AM, thierry blind wrote:
Hi Bart,
I killed my eclipse session yesterday evening, it was looping ;)
I've attached my logs ! There are full of errors :S
Thierry.
From:
bartlomie...@zend.com
To:
pdt-dev@eclipse.org
Date: Tue, 3 May 2016 18:01:36 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Hi Guys,
First of all thank you Thierry for spending some time on testing new indexer.
The problem that you found doesn't look good as you already said. I will try to
find the root cause as fast as possible. Could you possibly attach your entire
.log file?
Greetings,
Bart
On 5/3/2016 7:23 PM, thierry blind wrote:
Hi Kaloyan,
I've restarted everything (no more errors for now), first time it was probably
my fault (I had an eclipse zombie process when I started a new eclipse
instance).
I'm working on an existing workspace for now.
But there is still something very very wrong here.
I have very big projects (mainly because I use ORMs like Propel to generate a
lot of PHP classes), and it seems that the indexer loops over same (Propel)
files again and again.
The size of the "org.eclipse.dltk.core.index.lucene" directory is much bigger
than the "org.eclipse.dltk.core.index.sql.h2 directory" !!!!
And indexing is still not ended... I will probably stop it before it fills my
hard drive.
I will continue tests tomorrow, sadly tomorrow will be the last day I can do
some tests before next 2-3 weeks :/
Thierry.
From:
kaloya...@zend.com
To:
pdt-dev@eclipse.org
Date: Tue, 3 May 2016 16:44:15 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Hi Thierry,
Does the same happen in a fresh new workspace?
Kaloyan
From:
pdt-dev-boun...@eclipse.org
<pdt-dev-boun...@eclipse.org> on behalf of thierry blind
<thierrybl...@msn.com>
Sent: Tuesday, May 3, 2016 7:29 PM
To: PDT Developers
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Hi Kaloyan,
I just updated to latest PDT snapshot 4.0.0.201605031251 (from
http://download.eclipse.org/tools/pdt/updates/latest-nightly),
but I can't get the Lucene indexer to work.
Before filling a bug report I wanted to know what I do wrong:
1) I deleted .metadata\.plugins\org.eclipse.dltk.core.index.sql.h2 from my
workspace
2) I started eclipse, now I get an infinite loop of following errors:
eclipse.buildId=4.5.2.M20160212-1500
java.version=1.8.0_66
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=fr_FR
Framework arguments: -product org.eclipse.epp.package.jee.product
Command-line arguments: -os win32 -ws win32 -arch x86 -product
org.eclipse.epp.package.jee.product
org.eclipse.dltk.core.index.lucene
Error
Tue May 03 18:16:00 CEST 2016
Lock obtain timed out:
NativeFSLock@C:\workspace\.metadata\.plugins\org.eclipse.dltk.core.index.lucene\index\bc981875-9762-4936-b1d8-fbd9b4d8d2a3\timestamps\write.lock
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out:
NativeFSLock@C:\regroup\workspace10\.metadata\.plugins\org.eclipse.dltk.core.index.lucene\index\bc981875-9762-4936-b1d8-fbd9b4d8d2a3\timestamps\write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:780)
at
org.eclipse.dltk.internal.core.index.lucene.IndexContainer.getTimestampsWriter(IndexContainer.java:128)
at
org.eclipse.dltk.internal.core.index.lucene.IndexContainer.delete(IndexContainer.java:193)
at
org.eclipse.dltk.internal.core.index.lucene.LuceneManager.delete(LuceneManager.java:225)
at
org.eclipse.dltk.internal.core.index.lucene.LuceneIndexer.indexDocument(LuceneIndexer.java:161)
at
org.eclipse.dltk.internal.core.index2.SourceModulesRequest.run(SourceModulesRequest.java:81)
at
org.eclipse.dltk.core.search.indexing.AbstractJob.execute(AbstractJob.java:80)
at
org.eclipse.dltk.internal.core.search.processing.JobManager.run(JobManager.java:473)
at java.lang.Thread.run(Thread.java:745)
-------------------------------------------------------------------------------------------------------------
eclipse.buildId=4.5.2.M20160212-1500
java.version=1.8.0_66
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=fr_FR
Framework arguments: -product org.eclipse.epp.package.jee.product
Command-line arguments: -os win32 -ws win32 -arch x86 -product
org.eclipse.epp.package.jee.product
org.eclipse.dltk.core.index.lucene
Error
Tue May 03 18:15:42 CEST 2016
null
java.lang.NullPointerException
at
org.eclipse.dltk.internal.core.index.lucene.IndexContainer.delete(IndexContainer.java:193)
at
org.eclipse.dltk.internal.core.index.lucene.LuceneManager.delete(LuceneManager.java:225)
at
org.eclipse.dltk.internal.core.index.lucene.LuceneIndexer.indexDocument(LuceneIndexer.java:161)
at
org.eclipse.dltk.internal.core.index2.SourceModulesRequest.run(SourceModulesRequest.java:81)
at
org.eclipse.dltk.core.search.indexing.AbstractJob.execute(AbstractJob.java:80)
at
org.eclipse.dltk.internal.core.search.processing.JobManager.run(JobManager.java:473)
at java.lang.Thread.run(Thread.java:745)
-------------------------------------------------------------------------------------------------------------
Thierry.
From:
thierrybl...@msn.com
To:
pdt-dev@eclipse.org
Date: Tue, 3 May 2016 15:17:41 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Hope they enjoy their holidays, I will have mine next week ;)
I will test the Lucene indexer with great curiosity ;)
Thierry.
From:
kaloya...@zend.com
To:
pdt-dev@eclipse.org
Date: Tue, 3 May 2016 15:07:13 +0000
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Thanks, Thierry!
I guess the rest are enjoying the public holiday in Poland and are not working
hard on PDT
I went ahead and copied the latest nightly to this location:
http://download.eclipse.org/tools/pdt/updates/4.0-m7
I am pushed it to the SimRel: https://git.eclipse.org/r/71914/
Kaloyan
From:
pdt-dev-boun...@eclipse.org
<pdt-dev-boun...@eclipse.org> on behalf of thierry blind
<thierrybl...@msn.com>
Sent: Tuesday, May 3, 2016 5:07:26 PM
To: PDT Developers
Subject: Re: [pdt-dev] 4.0-M7 Milestone
Hi,
for me it should be ok ;)
From:
kaloya...@zend.com
To:
pdt-dev@eclipse.org
Date: Tue, 3 May 2016 13:44:10 +0000
Subject: [pdt-dev] 4.0-M7 Milestone
Hi,
I'd like to declare the 4.0-M7 milestone and go through the SimRel validation
to ensure that everything is correct with the new Lucene indexer.
Any last minute changes?
Kaloyan
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________ pdt-dev mailing list
pdt-dev@eclipse.org To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
_______________________________________________
pdt-dev mailing list
pdt-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/pdt-dev
