Re: lucene index deleted on restart
Hi,
With the little knowledge of Java I made the following modification in
org.apache.james.modules.mailbox.LuceneSearchMailboxModule
# add an extra includes
import com.google.inject.name.Names;
#add an additional line in the configure method
bind(Boolean.class).annotatedWith(Names.named("dropIndexOnStart")).toInstance(false);
These changes apparently solve the problem, but I don't think it should be
solved that way.
Best regards!
> On 20 Jul 2023, at 08:45, Mihai Zamfira
> wrote:
>
> Dear Benoit,
> Deleting indexes has a significant impact for some IMAP clients like Iphone
> Mail (standard email client on iphone).
> This application uses indexes in the IMAP synchronization process. I found
> that if the indexes are deleted, the users who use the Mail application on
> iPhone will only be able to access the emails that are found in the newly
> created indexes when the James server is started. All other emails that are
> in the email repository and are not indexed become inaccessible from the
> mobile phone
> The workaround is to run a full reindexing process after each startup of the
> James server, which consumes time and is not optimal
>
> Best regards
>
>
>> On 20 Jul 2023, at 05:32, Benoit TELLIER wrote:
>>
>> Hello Mihai,
>>
>> I reproduced the behaviour you mentionned on my side.
>>
>> - Start the demo image
>> - append a few mails
>>
>> List files. There is some Lucene files:
>>
>>
>>
>> -rw-r--r-- 1 root root 3.1K Jul 20 02:10 _k.fdt
>> -rw-r--r-- 1 root root 308 Jul 20 02:10 _k.fdx
>> -rw-r--r-- 1 root root 871 Jul 20 02:10 _k.fnm
>> -rw-r--r-- 1 root root 6.1K Jul 20 02:10 _k.frq
>> -rw-r--r-- 1 root root 1.6K Jul 20 02:10 _k.nrm
>> -rw-r--r-- 1 root root 4.5K Jul 20 02:10 _k.prx
>> -rw-r--r-- 1 root root 145 Jul 20 02:10 _k.tii
>> -rw-r--r-- 1 root root 9.5K Jul 20 02:10 _k.tisDo a search:
>>
>> $ openssl s_client -host 127.0.0.1 -port 993
>> [...]
>> a0 LOGIN user02@james.local 1234
>> a0 OK LOGIN completed.
>> a1 SELECT Drafts
>> [...]
>> a3 SEARCH SEEN
>> * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
>> * 1 FETCH (FLAGS (\Draft \Seen))
>> * 2 FETCH (FLAGS (\Draft \Seen))
>> * 3 FETCH (FLAGS (\Draft \Seen))
>> * 19 FETCH (FLAGS (\Draft \Seen))
>> a3 OK SEARCH completed.
>> - Restart James
>>
>> Then I noticed:
>>
>> - Lucene files disappeared
>> - Search no longer yield results
>>
>> $ openssl s_client -host 127.0.0.1 -port 993
>> [...]
>> a0 LOGIN user02@james.local 1234
>> a0 OK LOGIN completed.
>> a1 SELECT Drafts
>> [...]
>> a3 SEARCH SEEN
>> a3 OK SEARCH completed.
>> According to me, deletion of the Lucene files caused the alteration of the
>> search results.
>> I do not understand yet what caused the files to disapear, and is looking
>> for "why".
>>
>> --
>>
>>
>> Best regards,
>>
>>
>>
>> Benoit TELLIER
>>
>>
>>
>> General manager of Linagora VIETNAM.
>>
>> Product owner for Team-Mail product.
>>
>> Chairman of the Apache James project.
>>
>>
>>
>> Mail: btell...@linagora.com
>>
>> Tel: (0033) 6 77 26 04 58 (WhatsApp, Signal)
>>
>> On Jul 19, 2023 7:33 PM, from Mihai Zamfira hello,
>> on apache james 3.8.0 Jpa-guice with external rdbms i notice that when the
>> Apache James server is restarted, the lucene indexes are deleted.
>> is this normal behavior?
>>
>> Thanks!
>>
>>
>>
>> -
>> To unsubscribe, e-mail: > href="mailto:server-user-unsubscr...@james.apache.org";>server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: > href="mailto:server-user-h...@james.apache.org";>server-user-h...@james.apache.org
>>
>>
>
-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
Re: server-user Digest of: get.24132
Hello, I understand your opinion and I respect it, but this problem affects approximately 30% of the market share of mobile phone users. I will try to find a solution. > On 20 Jul 2023, at 14:15, Benoit TELLIER wrote: > > No this setting of the Lucene search index is not available yet. > > We could welcome a contribution on that topic, I believe. > > But having restrictive defaults seems like a good idea to me. > > -- > > > Best regards, > > > > Benoit TELLIER > > > > General manager of Linagora VIETNAM. > > Product owner for Team-Mail product. > > Chairman of the Apache James project. > > > > Mail: btell...@linagora.com > > Tel: (0033) 6 77 26 04 58 (WhatsApp, Signal) > > > On Jul 20, 2023 2:23 PM, from Mihai Zamfira Hi, > This behavior is specific to the Mail application on iPhone devices. If you > can guide me on where to intervene in the Apache James code to add a logger > to obtain the requested data. > However, we cannot modify the client application. > Regards > >> >> >> From: Benoit TELLIER >> Subject: Re: Lucene "maxClauseCount is set to 1024" error >> Date: 18 July 2023 at 09:51:38 EEST >> To: James Users List >> >> >> Hello, >> >> The IMAP clients likely transmit a too complex search query with to= >> o much clauses and Lucene refuses to execute such a complex query=2E >> >> This = >> is what you want! We want the search engine to be protected from denial of = >> service attacks: this might have been what you noticed=2E >> >> I'm interested b= >> y the IMAP contextual logs especially the MUA declared by the ID command, t= >> he exact search query that triggered the error=2E >> >> -- >> >> >> Best regards, >> >> >> >> Be= >> noit TELLIER >> >> >> >> General manager of Linagora VIETNAM=2E >> >> Product owner for T= >> eam-Mail product=2E >> >> Chairman of the Apache James project=2E >> >> >> >> Mail: btell= >> ier@linagora=2Ecom >> >> Tel: (0033) 6 77 26 04 58 (WhatsApp, Signal) >> >> >> On Jul 1= >> 8, 2023 12:04 PM, from Mihai Zamfira Hello=2E >> I use the latest version avai= >> lable on github of apache james jpa-guice which is integrated with LDAP and= >> an external rdbms=2E >> I get the following errors when mobile clients access= >> the account via IMAP=2E The emails are downloaded to the phone, but in the= >> log there are errors of the form below=2E >> I don't know if this is specific= >> only to the Iphone or not=2E >> >> >> 07:51:39=2E182 [ERROR] o=2Ea=2Ej=2Ei=2Ep= >> =2EAbstractMailboxProcessor - Unexpected error during IMAP processing >> org= >> =2Eapache=2Elucene=2Esearch=2EBooleanQuery$TooManyClauses: maxClauseCount i= >> s set to 1024 >>at org=2Eapache=2Elucene=2Esearch=2EBooleanQuery=2Eadd(Boole= >> anQuery=2Ejava:136) >>at org=2Eapache=2Elucene=2Esearch=2EBooleanQuery=2Eadd= >> (BooleanQuery=2Ejava:127) >>at org=2Eapache=2Ejames=2Emailbox=2Elucene=2Esea= >> rch=2ELuceneMessageSearchIndex=2EcreateUidQuery(LuceneMessageSearchIndex=2E= >> java:949) >>at org=2Eapache=2Ejames=2Emailbox=2Elucene=2Esearch=2ELuceneMess= >> ageSearchIndex=2EcreateFlagQuery(LuceneMessageSearchIndex=2Ejava:1023) >>at = >> org=2Eapache=2Ejames=2Emailbox=2Elucene=2Esearch=2ELuceneMessageSearchIndex= >> =2EcreateQuery(LuceneMessageSearchIndex=2Ejava:1208) >>at org=2Eapache=2Ejam= >> es=2Emailbox=2Elucene=2Esearch=2ELuceneMessageSearchIndex=2EsearchMultimap(= >> LuceneMessageSearchIndex=2Ejava:523) >>at org=2Eapache=2Ejames=2Emailbox=2El= >> ucene=2Esearch=2ELuceneMessageSearchIndex=2EdoSearch(LuceneMessageSearchInd= >> ex=2Ejava:491) >>at org=2Eapache=2Ejames=2Emailbox=2Estore=2Esearch=2EListen= >> ingMessageSearchIndex=2Elambda$search$2(ListeningMessageSearchIndex=2Ejava:= >> 84) >>at com=2Egithub=2Efge=2Elambdas=2Esupplier=2ESupplierChainer=2Elambda$= >> sneakyThrow$195(SupplierChainer=2Ejava:74) >>at java=2Ebase/java=2Eutil=2EOp= >> tional=2EorElseGet(Optional=2Ejava:369) >>at org=2Eapache=2Ejames=2Emailbox= >> =2Estore=2Esearch=2EListeningMessageSearchIndex=2Esearch(ListeningMessageSe= >> archIndex=2Ejava:84) >>at org=2Eapache=2Ejames=2Emailbox=2Estore=2EStoreMess= >> ageManager=2Esearch(StoreMessageManager=2Ejava:837) >>at org=2Eapache=2Ejame= >> s=2Emailbox=2Estore=2EStoreMessageManage
Re: server-user Digest of: get.24132
Hi, This behavior is specific to the Mail application on iPhone devices. If you can guide me on where to intervene in the Apache James code to add a logger to obtain the requested data. However, we cannot modify the client application. Regards > > > From: Benoit TELLIER > Subject: Re: Lucene "maxClauseCount is set to 1024" error > Date: 18 July 2023 at 09:51:38 EEST > To: James Users List > > > Hello, > > The IMAP clients likely transmit a too complex search query with to= > o much clauses and Lucene refuses to execute such a complex query=2E > > This = > is what you want! We want the search engine to be protected from denial of = > service attacks: this might have been what you noticed=2E > > I'm interested b= > y the IMAP contextual logs especially the MUA declared by the ID command, t= > he exact search query that triggered the error=2E > > -- > > > Best regards, > > > > Be= > noit TELLIER > > > > General manager of Linagora VIETNAM=2E > > Product owner for T= > eam-Mail product=2E > > Chairman of the Apache James project=2E > > > > Mail: btell= > ier@linagora=2Ecom > > Tel: (0033) 6 77 26 04 58 (WhatsApp, Signal) > > > On Jul 1= > 8, 2023 12:04 PM, from Mihai Zamfira Hello=2E > I use the latest version avai= > lable on github of apache james jpa-guice which is integrated with LDAP and= > an external rdbms=2E > I get the following errors when mobile clients access= > the account via IMAP=2E The emails are downloaded to the phone, but in the= > log there are errors of the form below=2E > I don't know if this is specific= > only to the Iphone or not=2E > > > 07:51:39=2E182 [ERROR] o=2Ea=2Ej=2Ei=2Ep= > =2EAbstractMailboxProcessor - Unexpected error during IMAP processing > org= > =2Eapache=2Elucene=2Esearch=2EBooleanQuery$TooManyClauses: maxClauseCount i= > s set to 1024 > at org=2Eapache=2Elucene=2Esearch=2EBooleanQuery=2Eadd(Boole= > anQuery=2Ejava:136) > at org=2Eapache=2Elucene=2Esearch=2EBooleanQuery=2Eadd= > (BooleanQuery=2Ejava:127) > at org=2Eapache=2Ejames=2Emailbox=2Elucene=2Esea= > rch=2ELuceneMessageSearchIndex=2EcreateUidQuery(LuceneMessageSearchIndex=2E= > java:949) > at org=2Eapache=2Ejames=2Emailbox=2Elucene=2Esearch=2ELuceneMess= > ageSearchIndex=2EcreateFlagQuery(LuceneMessageSearchIndex=2Ejava:1023) > at = > org=2Eapache=2Ejames=2Emailbox=2Elucene=2Esearch=2ELuceneMessageSearchIndex= > =2EcreateQuery(LuceneMessageSearchIndex=2Ejava:1208) > at org=2Eapache=2Ejam= > es=2Emailbox=2Elucene=2Esearch=2ELuceneMessageSearchIndex=2EsearchMultimap(= > LuceneMessageSearchIndex=2Ejava:523) > at org=2Eapache=2Ejames=2Emailbox=2El= > ucene=2Esearch=2ELuceneMessageSearchIndex=2EdoSearch(LuceneMessageSearchInd= > ex=2Ejava:491) > at org=2Eapache=2Ejames=2Emailbox=2Estore=2Esearch=2EListen= > ingMessageSearchIndex=2Elambda$search$2(ListeningMessageSearchIndex=2Ejava:= > 84) > at com=2Egithub=2Efge=2Elambdas=2Esupplier=2ESupplierChainer=2Elambda$= > sneakyThrow$195(SupplierChainer=2Ejava:74) > at java=2Ebase/java=2Eutil=2EOp= > tional=2EorElseGet(Optional=2Ejava:369) > at org=2Eapache=2Ejames=2Emailbox= > =2Estore=2Esearch=2EListeningMessageSearchIndex=2Esearch(ListeningMessageSe= > archIndex=2Ejava:84) > at org=2Eapache=2Ejames=2Emailbox=2Estore=2EStoreMess= > ageManager=2Esearch(StoreMessageManager=2Ejava:837) > at org=2Eapache=2Ejame= > s=2Emailbox=2Estore=2EStoreMessageManager=2Esearch(StoreMessageManager=2Eja= > va:132) > at org=2Eapache=2Ejames=2Eimap=2Eprocessor=2ESearchProcessor=2Eper= > formUidSearch(SearchProcessor=2Ejava:263) > at org=2Eapache=2Ejames=2Eimap= > =2Eprocessor=2ESearchProcessor=2Elambda$processRequestReactive$2(SearchProc= > essor=2Ejava:105) > at com=2Egithub=2Efge=2Elambdas=2Efunctions=2EFunctionCh= > ainer=2EdoApply(FunctionChainer=2Ejava:20) > at com=2Egithub=2Efge=2Elambdas= > =2Efunctions=2EThrowingFunction=2Eapply(ThrowingFunction=2Ejava:17) > at rea= > ctor=2Ecore=2Epublisher=2EMonoFlatMap$FlatMapMain=2EonNext(MonoFlatMap=2Eja= > va:132) > at reactor=2Ecore=2Epublisher=2EFluxMap$MapSubscriber=2EonNext(Flu= > xMap=2Ejava:122) > at reactor=2Ecore=2Epublisher=2EFluxOnErrorResume$ResumeS= > ubscriber=2EonNext(FluxOnErrorResume=2Ejava:79) > at reactor=2Ecore=2Epublis= > her=2EFluxSubscribeOnCallable$CallableSubscribeOnSubscription=2Erun(FluxSub= > scribeOnCallable=2Ejava:251) > at reactor=2Ecore=2Escheduler=2ESchedulerTask= > =2Ecall(SchedulerTask=2Ejava:68) > at reactor=2Ecore=2Escheduler=2EScheduler= > Task=2Ecall(SchedulerTask=2
Re: lucene index deleted on restart
Dear Benoit, Deleting indexes has a significant impact for some IMAP clients like Iphone Mail (standard email client on iphone). This application uses indexes in the IMAP synchronization process. I found that if the indexes are deleted, the users who use the Mail application on iPhone will only be able to access the emails that are found in the newly created indexes when the James server is started. All other emails that are in the email repository and are not indexed become inaccessible from the mobile phone The workaround is to run a full reindexing process after each startup of the James server, which consumes time and is not optimal Best regards > On 20 Jul 2023, at 05:32, Benoit TELLIER wrote: > > Hello Mihai, > > I reproduced the behaviour you mentionned on my side. > > - Start the demo image > - append a few mails > > List files. There is some Lucene files: > > > > -rw-r--r-- 1 root root 3.1K Jul 20 02:10 _k.fdt > -rw-r--r-- 1 root root 308 Jul 20 02:10 _k.fdx > -rw-r--r-- 1 root root 871 Jul 20 02:10 _k.fnm > -rw-r--r-- 1 root root 6.1K Jul 20 02:10 _k.frq > -rw-r--r-- 1 root root 1.6K Jul 20 02:10 _k.nrm > -rw-r--r-- 1 root root 4.5K Jul 20 02:10 _k.prx > -rw-r--r-- 1 root root 145 Jul 20 02:10 _k.tii > -rw-r--r-- 1 root root 9.5K Jul 20 02:10 _k.tisDo a search: > > $ openssl s_client -host 127.0.0.1 -port 993 > [...] > a0 LOGIN user02@james.local 1234 > a0 OK LOGIN completed. > a1 SELECT Drafts > [...] > a3 SEARCH SEEN > * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 > * 1 FETCH (FLAGS (\Draft \Seen)) > * 2 FETCH (FLAGS (\Draft \Seen)) > * 3 FETCH (FLAGS (\Draft \Seen)) > * 19 FETCH (FLAGS (\Draft \Seen)) > a3 OK SEARCH completed. > - Restart James > > Then I noticed: > > - Lucene files disappeared > - Search no longer yield results > > $ openssl s_client -host 127.0.0.1 -port 993 > [...] > a0 LOGIN user02@james.local 1234 > a0 OK LOGIN completed. > a1 SELECT Drafts > [...] > a3 SEARCH SEEN > a3 OK SEARCH completed. > According to me, deletion of the Lucene files caused the alteration of the > search results. > I do not understand yet what caused the files to disapear, and is looking for > "why". > > -- > > > Best regards, > > > > Benoit TELLIER > > > > General manager of Linagora VIETNAM. > > Product owner for Team-Mail product. > > Chairman of the Apache James project. > > > > Mail: btell...@linagora.com > > Tel: (0033) 6 77 26 04 58 (WhatsApp, Signal) > > On Jul 19, 2023 7:33 PM, from Mihai Zamfira hello, > on apache james 3.8.0 Jpa-guice with external rdbms i notice that when the > Apache James server is restarted, the lucene indexes are deleted. > is this normal behavior? > > Thanks! > > > > - > To unsubscribe, e-mail: href="mailto:server-user-unsubscr...@james.apache.org";>server-user-unsubscr...@james.apache.org > For additional commands, e-mail: href="mailto:server-user-h...@james.apache.org";>server-user-h...@james.apache.org > >
lucene index deleted on restart
hello, on apache james 3.8.0 Jpa-guice with external rdbms i notice that when the Apache James server is restarted, the lucene indexes are deleted. is this normal behavior? Thanks! - To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org For additional commands, e-mail: server-user-h...@james.apache.org
Re: Lucena throws java.io.IOException: null: NIOFSIndexInput
Hi, the server is an Red Hat Enterprise Linux release 8.8 (Ootpa) java -version java version "11.0.17" 2022-10-18 LTS Java(TM) SE Runtime Environment 18.9 (build 11.0.17+10-LTS-269) Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.17+10-LTS-269, mixed mode) Thanks. > On 19 Jul 2023, at 11:31, Benoit TELLIER wrote: > > Hello, > > Which operating system are you running James on? > > -- > > > Best regards, > > > > Benoit TELLIER > > > > General manager of Linagora VIETNAM. > > Product owner for Team-Mail product. > > Chairman of the Apache James project. > > > > Mail: btell...@linagora.com <mailto:btell...@linagora.com> > > Tel: (0033) 6 77 26 04 58 (WhatsApp, Signal) > > > Le juil. 19, 2023 11:25 AM, de Mihai Zamfira Hi, > on apache james JPA with external database which runs on Java 11 version we > face the below error. > This problem appears after the server runs for a period of time and leads to > desynchronization of lucene indexes. > I'm not a Java developer, but from my searches I understand that there is a > problem encountered when using NIOFSDirectory to interact with Lucenes in > Java 11 > Any advice is helpful, thanks > > 6:58:04.607 [ERROR] o.a.j.e.d.InVmEventDelivery - Error while processing > listener > java.nio.channels.ClosedChannelException: null > at > java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) > at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:790) > at > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162) > ... 67 common frames omitted > Wrapped by: java.io.IOException: null: > NIOFSIndexInput(path="/opt/james38_jpa/_251y.cfs") > at > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:180) > at > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:156) > at > org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:275) > at > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:270) > at > org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:40) > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:107) > at > org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:217) > at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:64) > at > org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133) > at > org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:174) > at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:202) > at > org.apache.lucene.index.TermInfosReader.terms(TermInfosReader.java:304) > at org.apache.lucene.index.SegmentReader.terms(SegmentReader.java:473) > at > org.apache.lucene.search.NumericRangeQuery$NumericRangeTermEnum.next(NumericRangeQuery.java:566) > at > org.apache.lucene.search.NumericRangeQuery$NumericRangeTermEnum.(NumericRangeQuery.java:508) > at > org.apache.lucene.search.NumericRangeQuery.getEnum(NumericRangeQuery.java:313) > at > org.apache.lucene.search.MultiTermQuery$RewriteMethod.getTermsEnum(MultiTermQuery.java:74) > at > org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:34) > at > org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:87) > at > org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:312) > at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:378) > at > org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:592) > at > org.apache.lucene.search.Searcher.createNormalizedWeight(Searcher.java:167) > at > org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:664) > at > org.apache.lucene.search.QueryWrapperFilter.getDocIdSet(QueryWrapperFilter.java:47) > at > org.apache.lucene.index.BufferedDeletesStream.applyQueryDeletes(BufferedDeletesStream.java:379) > at > org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:249) > at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3615) > at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3552) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:450) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:399) > at org.apache.lucene.index.IndexReader.open(IndexReader.java:2
Re: Lucena throws java.io.IOException: null: NIOFSIndexInput
I forgot to mention that it is version 3.8.0 of apache james > On 19 Jul 2023, at 07:24, Mihai Zamfira > wrote: > > Hi, > on apache james JPA with external database which runs on Java 11 version we > face the below error. > This problem appears after the server runs for a period of time and leads to > desynchronization of lucene indexes. > I'm not a Java developer, but from my searches I understand that there is a > problem encountered when using NIOFSDirectory to interact with Lucenes in > Java 11 > Any advice is helpful, thanks > > 6:58:04.607 [ERROR] o.a.j.e.d.InVmEventDelivery - Error while processing > listener > java.nio.channels.ClosedChannelException: null > at > java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) > at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:790) > at > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162) > ... 67 common frames omitted > Wrapped by: java.io.IOException: null: > NIOFSIndexInput(path="/opt/james38_jpa/_251y.cfs") > at > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:180) > at > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:156) > at > org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:275) > at > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:270) > at > org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:40) > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:107) > at > org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:217) > at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:64) > at > org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133) > at > org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:174) > at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:202) > at > org.apache.lucene.index.TermInfosReader.terms(TermInfosReader.java:304) > at org.apache.lucene.index.SegmentReader.terms(SegmentReader.java:473) > at > org.apache.lucene.search.NumericRangeQuery$NumericRangeTermEnum.next(NumericRangeQuery.java:566) > at > org.apache.lucene.search.NumericRangeQuery$NumericRangeTermEnum.(NumericRangeQuery.java:508) > at > org.apache.lucene.search.NumericRangeQuery.getEnum(NumericRangeQuery.java:313) > at > org.apache.lucene.search.MultiTermQuery$RewriteMethod.getTermsEnum(MultiTermQuery.java:74) > at > org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:34) > at > org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:87) > at > org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:312) > at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:378) > at > org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:592) > at > org.apache.lucene.search.Searcher.createNormalizedWeight(Searcher.java:167) > at > org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:664) > at > org.apache.lucene.search.QueryWrapperFilter.getDocIdSet(QueryWrapperFilter.java:47) > at > org.apache.lucene.index.BufferedDeletesStream.applyQueryDeletes(BufferedDeletesStream.java:379) > at > org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:249) > at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3615) > at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3552) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:450) > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:399) > at org.apache.lucene.index.IndexReader.open(IndexReader.java:296) > at > org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.update(LuceneMessageSearchIndex.java:1259) > at > org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.lambda$update$4(LuceneMessageSearchIndex.java:1253) > at > com.github.fge.lambdas.runnable.RunnableChainer.doRun(RunnableChainer.java:18) > at > com.github.fge.lambdas.runnable.ThrowingRunnable.run(ThrowingRunnable.java:16) > ... 32 common frames omitted > Wrapped by: com.github.fge.lambdas.ThrownByLambdaException: > java.io.IOException: null: NIOFSIndexInput(path="/opt/james38_jpa/_251y.cfs") > at > com.github.fge.lambdas.runnable.ThrowingRunna
Lucena throws java.io.IOException: null: NIOFSIndexInput
Hi, on apache james JPA with external database which runs on Java 11 version we face the below error. This problem appears after the server runs for a period of time and leads to desynchronization of lucene indexes. I'm not a Java developer, but from my searches I understand that there is a problem encountered when using NIOFSDirectory to interact with Lucenes in Java 11 Any advice is helpful, thanks 6:58:04.607 [ERROR] o.a.j.e.d.InVmEventDelivery - Error while processing listener java.nio.channels.ClosedChannelException: null at java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:790) at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:162) ... 67 common frames omitted Wrapped by: java.io.IOException: null: NIOFSIndexInput(path="/opt/james38_jpa/_251y.cfs") at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:180) at org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:156) at org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:275) at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:270) at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:40) at org.apache.lucene.store.DataInput.readVInt(DataInput.java:107) at org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:217) at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:64) at org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:133) at org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:174) at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:202) at org.apache.lucene.index.TermInfosReader.terms(TermInfosReader.java:304) at org.apache.lucene.index.SegmentReader.terms(SegmentReader.java:473) at org.apache.lucene.search.NumericRangeQuery$NumericRangeTermEnum.next(NumericRangeQuery.java:566) at org.apache.lucene.search.NumericRangeQuery$NumericRangeTermEnum.(NumericRangeQuery.java:508) at org.apache.lucene.search.NumericRangeQuery.getEnum(NumericRangeQuery.java:313) at org.apache.lucene.search.MultiTermQuery$RewriteMethod.getTermsEnum(MultiTermQuery.java:74) at org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:34) at org.apache.lucene.search.ConstantScoreAutoRewrite.rewrite(ConstantScoreAutoRewrite.java:87) at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:312) at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:378) at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:592) at org.apache.lucene.search.Searcher.createNormalizedWeight(Searcher.java:167) at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:664) at org.apache.lucene.search.QueryWrapperFilter.getDocIdSet(QueryWrapperFilter.java:47) at org.apache.lucene.index.BufferedDeletesStream.applyQueryDeletes(BufferedDeletesStream.java:379) at org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:249) at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3615) at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3552) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:450) at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:399) at org.apache.lucene.index.IndexReader.open(IndexReader.java:296) at org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.update(LuceneMessageSearchIndex.java:1259) at org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.lambda$update$4(LuceneMessageSearchIndex.java:1253) at com.github.fge.lambdas.runnable.RunnableChainer.doRun(RunnableChainer.java:18) at com.github.fge.lambdas.runnable.ThrowingRunnable.run(ThrowingRunnable.java:16) ... 32 common frames omitted Wrapped by: com.github.fge.lambdas.ThrownByLambdaException: java.io.IOException: null: NIOFSIndexInput(path="/opt/james38_jpa/_251y.cfs") at com.github.fge.lambdas.runnable.ThrowingRunnable.run(ThrowingRunnable.java:20) at reactor.core.publisher.MonoRunnable.call(MonoRunnable.java:73) at reactor.core.publisher.MonoRunnable.call(MonoRunnable.java:32) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:252) at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51) at reactor.core.publisher.MonoUsing.subscribe(MonoUsing.java:109) at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:53)
Lucene "maxClauseCount is set to 1024" error
Hello. I use the latest version available on github of apache james jpa-guice which is integrated with LDAP and an external rdbms. I get the following errors when mobile clients access the account via IMAP. The emails are downloaded to the phone, but in the log there are errors of the form below. I don't know if this is specific only to the Iphone or not. 07:51:39.182 [ERROR] o.a.j.i.p.AbstractMailboxProcessor - Unexpected error during IMAP processing org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024 at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:136) at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:127) at org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.createUidQuery(LuceneMessageSearchIndex.java:949) at org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.createFlagQuery(LuceneMessageSearchIndex.java:1023) at org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.createQuery(LuceneMessageSearchIndex.java:1208) at org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.searchMultimap(LuceneMessageSearchIndex.java:523) at org.apache.james.mailbox.lucene.search.LuceneMessageSearchIndex.doSearch(LuceneMessageSearchIndex.java:491) at org.apache.james.mailbox.store.search.ListeningMessageSearchIndex.lambda$search$2(ListeningMessageSearchIndex.java:84) at com.github.fge.lambdas.supplier.SupplierChainer.lambda$sneakyThrow$195(SupplierChainer.java:74) at java.base/java.util.Optional.orElseGet(Optional.java:369) at org.apache.james.mailbox.store.search.ListeningMessageSearchIndex.search(ListeningMessageSearchIndex.java:84) at org.apache.james.mailbox.store.StoreMessageManager.search(StoreMessageManager.java:837) at org.apache.james.mailbox.store.StoreMessageManager.search(StoreMessageManager.java:132) at org.apache.james.imap.processor.SearchProcessor.performUidSearch(SearchProcessor.java:263) at org.apache.james.imap.processor.SearchProcessor.lambda$processRequestReactive$2(SearchProcessor.java:105) at com.github.fge.lambdas.functions.FunctionChainer.doApply(FunctionChainer.java:20) at com.github.fge.lambdas.functions.ThrowingFunction.apply(ThrowingFunction.java:17) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:132) at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) at reactor.core.publisher.FluxSubscribeOnCallable$CallableSubscribeOnSubscription.run(FluxSubscribeOnCallable.java:251) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) - To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org For additional commands, e-mail: server-user-h...@james.apache.org
Re: Jdbc support
Thanks Benoit, I want to migrate emails from Cassandra to an external RDBMS. Can you provide me with some guidance on the reprocessing emails to get started? I will create a small document on this topic. Thank you. > On 12 Jul 2023, at 18:59, Benoit TELLIER wrote: > > Hello Mihai > > RDBMS is supported through JPAMailRepository (available from 3.8.0 onward). > > Migration without data loss can easily be done thanks to reprocessing. > > -- > > > Best regards, > > > > Benoit TELLIER > > > > General manager of Linagora VIETNAM. > > Product owner for Team-Mail product. > > Chairman of the Apache James project. > > > > Mail: btell...@linagora.com > > Tel: (0033) 6 77 26 04 58 (WhatsApp, Signal) > > > --- Forwarded message --- > Subject: Jdbc support > Date: Jul 12, 2023 6:55 PM > From: Mihai Zamfira > To: gene...@james.apache.org > Hi, I saw that the JDBC Repository is deprecated. > It is not clear to me the reason and whether it is possible to use an rdbms > as an email container. > Thank you! - To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org For additional commands, e-mail: server-user-h...@james.apache.org
Sieve mailet error
Hi,
im using a james 3.8 with cassandra and opensearch.
my “local-delivery” processor configuration looks like below .
ignore
ignore
ignore
The problem appears when an email with 2 or more local recipients it’s received
and Sieve it throw the following error "Invalid number of recipients - 2.
Exactly 1 recipient is expected. Moreover, the email is received by the
recipient and duplicated with another email in which the SIEVE error is
specified
Any help is appreciated.
Below i added first lines from the error stack
09:23:37.239 [INFO ] o.a.j.s.SendMailHandler - Successfully spooled mail
Mail1686464617103-df62f02b-fc93-45e0-901c-b5a206bc82af with messageId
<657624c0-3cf9-4f37-93e0-1cbeacfd6...@logika.ro> from test.us...@domain.com on
79.138.136.47/79.138.136.47 for [test.us...@domain.com, test.us...@domain.com]
09:23:37.239 [DEBUG] o.a.j.s.DataLineJamesMessageHookHandler - executing hook
org.apache.james.protocols.smtp.core.log.HookResultLogger@5f33b30e
09:23:37.239 [DEBUG] o.a.j.p.s.c.l.HookResultLogger -
org.apache.james.smtpserver.SendMailHandler: result= (OK CONNECTED)
09:23:37.239 [DEBUG] o.a.j.s.DataLineJamesMessageHookHandler - executing hook
org.apache.james.smtpserver.jmx.HookResultJMXMonitor@571441c2
09:23:37.248 [DEBUG] o.a.j.m.i.JamesMailSpooler - Begin processing mail
Mail1686464617103-df62f02b-fc93-45e0-901c-b5a206bc82af
09:23:37.249 [DEBUG] o.a.j.m.l.AbstractStateCompositeProcessor - Call
MailProcessor root
09:23:37.258 [DEBUG] o.a.j.m.l.AbstractStateCompositeProcessor - Call
MailProcessor transport
09:23:37.283 [DEBUG] o.a.j.m.l.AbstractStateCompositeProcessor - Call
MailProcessor local-delivery
09:23:37.289 [DEBUG] o.a.j.i.d.m.DefaultImapDecoder - Got :
Tag{value=87.241}
09:23:37.289 [DEBUG] o.a.j.i.d.m.DefaultImapDecoder - Got : SELECT
09:23:37.306 [DEBUG] o.a.j.m.c.GhostMailbox - Read mailbox succeeded
09:23:37.308 [DEBUG] o.a.j.m.s.StoreMailboxManager - Loaded mailbox
#private:test.us...@domain.com:Sent Messages
09:23:37.314 [DEBUG] o.a.j.t.m.j.d.SieveExecutor - Evaluating
org.apache.james.transport.mailets.jsieve.SieveMailAdapter Envelope From:
test.us...@domain.com Envelope To: test.us...@domain.com Message ID:
<657624c0-3cf9-4f37-93e0-1cbeacfd6...@logika.ro> against
""
09:23:37.339 [INFO ] o.a.j.t.m.j.SieveMailAdapter - Executing action: Action:
org.apache.jsieve.mail.ActionKeep
09:23:37.340 [ERROR] o.a.j.SieveFactory - Evaluation failed.
javax.mail.MessagingException: Invalid number of recipients - 2. Exactly 1
recipient is expected.
at
org.apache.james.transport.mailets.jsieve.ActionUtils.getSoleRecipient(ActionUtils.java:52)
at
org.apache.james.transport.mailets.jsieve.FileIntoAction.execute(FileIntoAction.java:76)
at
org.apache.james.transport.mailets.jsieve.KeepAction.execute(KeepAction.java:63)
at
org.apache.james.transport.mailets.jsieve.KeepAction.execute(KeepAction.java:42)
at
org.apache.james.transport.mailets.jsieve.ActionDispatcher.execute(ActionDispatcher.java:70)
at
org.apache.james.transport.mailets.jsieve.SieveMailAdapter.executeActions(SieveMailAdapter.java:160)
... 97 common frames omitted
Wrapped by: org.apache.jsieve.exception.SieveException:
javax.mail.MessagingException: Invalid number of recipients - 2. Exactly 1
recipient is expected.
at
org.apache.james.transport.mailets.jsieve.SieveMailAdapter.executeActions(SieveMailAdapter.java:162)
at org.apache.jsieve.SieveFactory.evaluate(SieveFactory.java:165)
at
org.apache.james.transport.mailets.jsieve.delivery.SieveExecutor.sieveMessageEvaluate(SieveExecutor.java:138)
at
org.apache.james.transport.mailets.jsieve.delivery.SieveExecutor.sieveMessage(SieveExecutor.java:118)
at
org.apache.james.transport.mailets.jsieve.delivery.SieveExecutor.execute(SieveExecutor.java:112)
at org.apache.james.transport.mailets.Sieve.service(Sieve.java:71)
at
org.apache.james.mailetcontainer.impl.ProcessorImpl.process(ProcessorImpl.java:81)
at
com.github.fge.lambdas.consumers.ConsumerChainer.lambda$sneakyThrow$9(ConsumerChainer.java:73)
-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
WebAdmin encountered an unexpected internal error
Hi,
I recently upgraded from 3.7.4 to james 3.8 .
The server reports warnings like below
"[WARN ] o.a.j.PeriodicalHealthChecks - DEGRADED: EventDeadLettersHealthCheck :
EventDeadLetters contain events. This might indicate transient failure on event
processing”
When I try to see the details of the events using the webadmin API "curl -XGET
http: //ip:port/events/deadLetter/groups" I get the following error as a
response
{
"statusCode": 500,
"type": "ServerError",
"message": "WebAdmin encountered an unexpected internal error",
"details": null
}
From the java trace I cannot understand why it's trying to use elasticsearch
when this is a new opensearch installation.
13:13:57.700 [INFO ] o.a.j.w.m.LoggingRequestFilter - WebAdmin request received
13:13:57.712 [ERROR] s.h.m.GeneralError -
java.lang.ClassNotFoundException:
org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex$ElasticSearchListeningMessageSearchIndexGroup
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at org.apache.james.events.Group.deserialize(Group.java:48)
... 39 common frames omitted
Wrapped by: org.apache.james.events.Group$GroupDeserializationException:
java.lang.ClassNotFoundException:
org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex$ElasticSearchListeningMessageSearchIndexGroup
at org.apache.james.events.Group.deserialize(Group.java:51)
at
org.apache.james.events.CassandraEventDeadLettersGroupDAO.lambda$retrieveAllGroups$0(CassandraEventDeadLettersGroupDAO.java:79)
at
com.github.fge.lambdas.functions.FunctionChainer.doApply(FunctionChainer.java:20)
at
com.github.fge.lambdas.functions.ThrowingFunction.apply(ThrowingFunction.java:17)
... 36 common frames omitted
Wrapped by: com.github.fge.lambdas.ThrownByLambdaException:
org.apache.james.events.Group$GroupDeserializationException:
java.lang.ClassNotFoundException:
org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex$ElasticSearchListeningMessageSearchIndexGroup
at
com.github.fge.lambdas.functions.ThrowingFunction.apply(ThrowingFunction.java:21)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)
at
com.datastax.dse.driver.internal.core.cql.reactive.ReactiveResultSetSubscription.doOnNext(ReactiveResultSetSubscription.java:364)
at
com.datastax.dse.driver.internal.core.cql.reactive.ReactiveResultSetSubscription.drain(ReactiveResultSetSubscription.java:236)
at
com.datastax.dse.driver.internal.core.cql.reactive.ReactiveResultSetSubscription.lambda$fetchNextPageAndEnqueue$2(ReactiveResultSetSubscription.java:358)
at
java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestHandler.setFinalResult(CqlRequestHandler.java:324)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestHandler.access$1500(CqlRequestHandler.java:95)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestHandler$NodeResponseCallback.onResponse(CqlRequestHandler.java:655)
at
com.datastax.oss.driver.internal.core.channel.InFlightHandler.channelRead(InFlightHandler.java:257)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at
io.
Input stream limit exceeded on elasticsearch
Hi,
I'm trying to configure an Apache James 3.7.4 that uses Cassandra,
ElasticSearch and OpenLDAP. When synchronizing emails, I get the following error
23:26:23.748 [WARN ] o.a.j.m.e.v.e.ElasticSearchListeningMessageSearchIndex -
Indexing mailbox Sent-b947fde0-e3a7-11ed-ade3-e76310c4a198 of user
some.u...@domain.com on message MessageUid{uid=2710} without attachments
java.io.IOException: Input stream limit exceeded. Limit was 104857600 and
position 104857600
at
org.apache.james.mime4j.io.LimitedInputStream.enforceLimit(LimitedInputStream.java:39)
at
org.apache.james.mime4j.io.LimitedInputStream.read(LimitedInputStream.java:51)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
at
org.apache.james.mime4j.stream.MimeEntity.advanceToBoundary(MimeEntity.java:393)
at
org.apache.james.mime4j.stream.MimeEntity.advance(MimeEntity.java:331)
at
org.apache.james.mime4j.stream.MimeTokenStream.next(MimeTokenStream.java:374)
at
org.apache.james.mailbox.elasticsearch.v7.json.MimePartParser.parse(MimePartParser.java:68)
at
org.apache.james.mailbox.elasticsearch.v7.json.IndexableMessage$Builder.instantiateIndexedMessage(IndexableMessage.java:101)
at
org.apache.james.mailbox.elasticsearch.v7.json.IndexableMessage$Builder.build(IndexableMessage.java:72)
... 64 common frames omitted
Wrapped by: java.lang.RuntimeException: java.io.IOException: Input stream limit
exceeded. Limit was 104857600 and position 104857600
at
org.apache.james.mailbox.elasticsearch.v7.json.IndexableMessage$Builder.build(IndexableMessage.java:74)
at
org.apache.james.mailbox.elasticsearch.v7.json.MessageToElasticSearchJson.convertToJson(MessageToElasticSearchJson.java:67)
at
org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex.generateIndexedJson(ElasticSearchListeningMessageSearchIndex.java:165)
at
org.apache.james.mailbox.elasticsearch.v7.events.ElasticSearchListeningMessageSearchIndex.lambda$add$0(ElasticSearchListeningMessageSearchIndex.java:158)
at reactor.core.publisher.MonoCallable.call(MonoCallable.java:92)
It is clearly that is a limit of 100Mb in the integration stream with
ElasticSearch, but I can't find how can increase it.
-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
Re: Cannot receive email for local users
Hi,
in my case, the problem was not generated by the wrong choice of the LDAP
attribute (uid or mail) because both have the value @.
After a long period of searching and debugging I discovered that the smtp works
as it should if in the “usersrepository.xml” contains the following parameter
in LDAP configuration " true”.
Please notice that it is called “supportsVirtualHosting", instead of
“enableVirtualHosting” as it is written in the effective documentation. Now,
my configuration contains both tags set to true and looks like below.
ldap://localhost:389";
principal="cn=domain_admin,dc=example,dc=com"
credentials=""
userBase="ou=users,ou=james,dc=example,dc=com"
userIdAttribute="uid"
userObjectClass="person">
true
true
true
Probably a developer can clarify why this difference is.
Regards!
> On 18 Apr 2023, at 04:17, Benoit TELLIER wrote:
>
> With "virtual hosting" turned on, the attribute should be an email address so
> "mail".
>
> We may need to make this more explicit in the documentation, as this might be
> a common mistake. Contribution welcome.
>
> Best regards,
>
> Benoit
>
> On 18/04/2023 00:27, Mihai Zamfira wrote:
>> Hi,
>> domain was created before
>> and the result it's below
>>
>> curl -XGET http://:/domains
>>
>> [
>> "example.com",
>> "localhost",
>> "127.0.0.1"
>> ]
>>
>> The problem has something to do with
>> "true” tag from
>> usersrepository.xml file because if i change it to “false” and the
>> userIdAttribute to an LDAP attribute that contain only the username (without
>> @example.com) everything works as expected.
>>
>> thanks.
>>
>>> On 17 Apr 2023, at 18:35, Tung Tran Van wrote:
>>>
>>> Hi,
>>>
>>> Did you add donain?
>>> Can you try:
>>>
>>> curl -XGET http://localhost:8000/ <http://localhost:8000/users>domains
>>>
>>>
>>> Vào Th 2, 17 thg 4, 2023 lúc 22:26 Mihai Zamfira
>>> đã viết:
>>>
>>>> Hi,
>>>> I have installed version 3.7.4 of Apache James on OEL8 and it works
>>>> integrated with Cassandra, Elasticsearch, SOGo (as web mail frontend) and
>>>> the users are stored in OpenLDAP.
>>>> LDAP configuration works and i can see users thru WebAdmin and also i'm
>>>> able to connect on IMAP fomr SOGO or others email clients.
>>>> The problem is with the SMTP server that refuses to recognize local users
>>>> reporting "550 5.1.1 Unknown user:"
>>>>
>>>> The content of the "usersrepository.xml" file looks like below:
>>>>
>>>> >>>ldapHost="ldap://localhost:389";
>>>>principal="cn=domain_admin,dc=example,dc=com"
>>>>credentials=“"
>>>>userBase="ou=users,ou=james,dc=example,dc=com"
>>>>userIdAttribute="uid"
>>>>userObjectClass="person">
>>>> true
>>>> true
>>>>
>>>>
>>>> Therefore the VirtualHosting it's enable and the user has ending with "@
>>>> example.com"
>>>>
>>>> resuts from "curl -XGET http://localhost:8000/users"; show me that the
>>>> Apache James can read users from OpenLdap
>>>>
>>>> [
>>>>{
>>>>"username": "test.u...@example.com"
>>>>}
>>>> ]
>>>>
>>>>
>>>> The real problem appears when I try to set the SMTP server with the telnet
>>>> command and the result is shown below
>>>>
>>>> [root@mail ~]# telnet 127.0.0.1 25
>>>> Trying 127.0.0.1...
>>>> Connected to 127.0.0.1.
>>>> Escape character is '^]'.
>>>> 220 Apache JAMES awesome SMTP Server
>>>> ehlo test
>>>> 250-mail.example.com Hello test [127.0.0.1])
>>>> 250-PIPELINING
>>>> 250-ENHANCEDSTATUSCODES
>>>> 250 8BITMIME
>>>> mail from:
>>>> 250 2.1.0 Sender OK
>>>> rcpt to:
>>>> 550 5.1.1 Unknown user: test.u...@example.com
>>>> Connection closed by foreign host.
>>>> -
>>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>>
>>>> --
>>> Tung, Tran Van
>>> *Phone:* (+84) 35 757 6258
>>
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>
>>
>
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
>
-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
Re: Cannot receive email for local users
Hi,
domain was created before
and the result it's below
curl -XGET http://:/domains
[
"example.com",
"localhost",
"127.0.0.1"
]
The problem has something to do with
"true” tag from
usersrepository.xml file because if i change it to “false” and the
userIdAttribute to an LDAP attribute that contain only the username (without
@example.com) everything works as expected.
thanks.
> On 17 Apr 2023, at 18:35, Tung Tran Van wrote:
>
> Hi,
>
> Did you add donain?
> Can you try:
>
> curl -XGET http://localhost:8000/ <http://localhost:8000/users>domains
>
>
> Vào Th 2, 17 thg 4, 2023 lúc 22:26 Mihai Zamfira
> đã viết:
>
>> Hi,
>> I have installed version 3.7.4 of Apache James on OEL8 and it works
>> integrated with Cassandra, Elasticsearch, SOGo (as web mail frontend) and
>> the users are stored in OpenLDAP.
>> LDAP configuration works and i can see users thru WebAdmin and also i'm
>> able to connect on IMAP fomr SOGO or others email clients.
>> The problem is with the SMTP server that refuses to recognize local users
>> reporting "550 5.1.1 Unknown user:"
>>
>> The content of the "usersrepository.xml" file looks like below:
>>
>> >ldapHost="ldap://localhost:389";
>>principal="cn=domain_admin,dc=example,dc=com"
>>credentials=“"
>>userBase="ou=users,ou=james,dc=example,dc=com"
>>userIdAttribute="uid"
>>userObjectClass="person">
>> true
>> true
>>
>>
>> Therefore the VirtualHosting it's enable and the user has ending with "@
>> example.com"
>>
>> resuts from "curl -XGET http://localhost:8000/users"; show me that the
>> Apache James can read users from OpenLdap
>>
>> [
>>{
>>"username": "test.u...@example.com"
>>}
>> ]
>>
>>
>> The real problem appears when I try to set the SMTP server with the telnet
>> command and the result is shown below
>>
>> [root@mail ~]# telnet 127.0.0.1 25
>> Trying 127.0.0.1...
>> Connected to 127.0.0.1.
>> Escape character is '^]'.
>> 220 Apache JAMES awesome SMTP Server
>> ehlo test
>> 250-mail.example.com Hello test [127.0.0.1])
>> 250-PIPELINING
>> 250-ENHANCEDSTATUSCODES
>> 250 8BITMIME
>> mail from:
>> 250 2.1.0 Sender OK
>> rcpt to:
>> 550 5.1.1 Unknown user: test.u...@example.com
>> Connection closed by foreign host.
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>
>> --
> Tung, Tran Van
> *Phone:* (+84) 35 757 6258
-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
Cannot receive email for local users
Hi,
I have installed version 3.7.4 of Apache James on OEL8 and it works integrated
with Cassandra, Elasticsearch, SOGo (as web mail frontend) and the users are
stored in OpenLDAP.
LDAP configuration works and i can see users thru WebAdmin and also i'm able to
connect on IMAP fomr SOGO or others email clients.
The problem is with the SMTP server that refuses to recognize local users
reporting "550 5.1.1 Unknown user:"
The content of the "usersrepository.xml" file looks like below:
ldap://localhost:389";
principal="cn=domain_admin,dc=example,dc=com"
credentials=“"
userBase="ou=users,ou=james,dc=example,dc=com"
userIdAttribute="uid"
userObjectClass="person">
true
true
Therefore the VirtualHosting it's enable and the user has ending with
"@example.com"
resuts from "curl -XGET http://localhost:8000/users"; show me that the Apache
James can read users from OpenLdap
[
{
"username": "test.u...@example.com"
}
]
The real problem appears when I try to set the SMTP server with the telnet
command and the result is shown below
[root@mail ~]# telnet 127.0.0.1 25
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
220 Apache JAMES awesome SMTP Server
ehlo test
250-mail.example.com Hello test [127.0.0.1])
250-PIPELINING
250-ENHANCEDSTATUSCODES
250 8BITMIME
mail from:
250 2.1.0 Sender OK
rcpt to:
550 5.1.1 Unknown user: test.u...@example.com
Connection closed by foreign host.
-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
