RE: Lucene index upgrade from 4.6 to 8 facing issue

[Jyothsna Bavisetti]

Hi All,

1. After Upgrading Lucene from 4.6 to 8, facing issue in search process.
2.We are creating 5 different folder for indexing with different index id ( 5 
folders from 5 different tables). During search process we will join all these 
data to display.  After upgrading we are facing issue in search data.
3. When we are applying filter for different fields from different index_id. It 
is searching for only one field. 
4. I am seeing difference in Join Query. 
JoinUtil.createJoinQuery(fromField, false, toField, Query, srch, 
ScoreMode.None); 
Query formation with different versions:
Lucene 4.6.0:
TermsQuery{field=case.id}
fromQuery=+(history.attribute:SExtendedAttribute1) 
+history.modifiedBy:3ff0  (different parameter)
Lucene 8.0.0:
TermsQuery{field=case.idfromQuery=+(history.attribute:SExtendedAttribute1) 
+history.modifiedBy:3ff0}

fromQuery=+(history.attribute:SExtendedAttribute1) 
+history.modifiedBy:3ff0  (different parameter is also visible.)

Please Suggest me , as new to the Lucene unable to predict it. 

-Original Message-
From: Jyothsna Bavisetti 
Sent: Wednesday, October 2, 2019 2:49 AM
To: dev@lucene.apache.org
Subject: RE: Lucene index upgrade from 4.6 to 8 facing issue

Thank you,


Thanks,
Jyothsna

Hi Shawn,

Any doc or links for re indexing process. We are using Lucene core 8.0.0.


Thanks,
Jyothsna

-Original Message-
From: Jyothsna Bavisetti
Sent: Thursday, September 26, 2019 11:51 PM
To: dev@lucene.apache.org
Subject: RE: Lucene index upgrade from 4.6 to 8 facing issue

Hi Shawn,

Re-indexing is costly transaction in my use case as it takes more than three 
days. Please let me know if any work around?

Thanks,
Jyothsna
-Original Message-
From: Shawn Heisey 
Sent: Thursday, September 26, 2019 11:35 PM
To: dev@lucene.apache.org
Subject: Re: Lucene index upgrade from 4.6 to 8 facing issue

On 9/26/2019 11:41 AM, Jyothsna Bavisetti wrote:
> I am trying to upgrade Lucene index from 4.6 to 8.0.0. When I'm trying 
> to upgrade tool using:
> 
> java -cp lucene-core.jar:lucene-backward-codecs.jar \
> 
> org.apache.lucene.index.IndexUpgrader-delete-prior-commits  \



> Please let me know any option other than reindexing.

If you're upgrading more than one major version, you must reindex. 
Multiple major version upgrades have always been discouraged and never 
guaranteed to work, but now such upgrades are explicitly denied.

When you used the IndexUpgrader from Lucene 6, the Lucene version was written 
into the index.  The recorded version was preserved by the upgrader for version 
7.  When the index was subsequently read by version 8, it complained because 
the original index was not written by version 7 or later.

Thanks,
Shawn

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional 
commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

forceMerge and unused metadata

[Shawn Heisey]

A question came across the #solr IRC channel, where the user was seeing 
fields in their /admin/luke endpoint about a bunch of fields they used 
to use, but are no longer in any current documents.  That URL endpoint 
provides information about the fields in the index, getting most of that 
info directly from Lucene.


I asked them to run an optimize (forceMerge in Lucene) and see what that 
did.  It did not remove those fields.


Discussing it with other Solr committers on the lucene-solr slack 
channel, this is apparently known -- a forceMerge does not eliminate any 
field metadata, even if the field is not referenced by any non-deleted 
document.


What I'm wondering is whether it would be possible to adjust merging so 
that it can determine what pieces of metadata (like field information) 
are unused in the index and remove them.  It would be fine if this were 
only an option on forceMerge, but nice if it were something that could 
happen on any merge.  That discussion on slack indicated that it might 
be prohibitively expensive to do this.  Can one of our experts on Lucene 
merging respond?


This particular user has no option that I am aware of other than to 
rebuild their index.  They're running version 4.2.1.


Thanks,
Shawn

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [lucene-solr] branch master updated: SOLR-13783: fix failing tests due to NamedList.toString() change

[Tomás Fernández Löbbe]

Thanks Munendra,
sorry for the noise.

On Tue, Oct 29, 2019 at 1:06 AM  wrote:

> This is an automated email from the ASF dual-hosted git repository.
>
> munendrasn pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
>  new b82b772  SOLR-13783: fix failing tests due to
> NamedList.toString() change
> b82b772 is described below
>
> commit b82b7725e110cd482c7a4372dc3b1a47eee2023a
> Author: Munendra S N 
> AuthorDate: Tue Oct 29 13:35:31 2019 +0530
>
> SOLR-13783: fix failing tests due to NamedList.toString() change
> ---
>  solr/core/src/test/org/apache/solr/core/TestBadConfig.java |  2 +-
>  .../update/processor/UpdateRequestProcessorFactoryTest.java| 10
> +-
>  2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/solr/core/src/test/org/apache/solr/core/TestBadConfig.java
> b/solr/core/src/test/org/apache/solr/core/TestBadConfig.java
> index db04152..1dfad85 100644
> --- a/solr/core/src/test/org/apache/solr/core/TestBadConfig.java
> +++ b/solr/core/src/test/org/apache/solr/core/TestBadConfig.java
> @@ -82,7 +82,7 @@ public class TestBadConfig extends
> AbstractBadConfigTestBase {
>
>public void testSchemaMutableButNotManaged() throws Exception {
>  assertConfigs("bad-solrconfig-schema-mutable-but-not-managed.xml",
> -  "schema-minimal.xml", "Unexpected arg(s):
> {mutable=false,managedSchemaResourceName=schema.xml}");
> +  "schema-minimal.xml", "Unexpected arg(s):
> {mutable=false, managedSchemaResourceName=schema.xml}");
>}
>
>public void testManagedSchemaCannotBeNamedSchemaDotXml() throws
> Exception {
> diff --git
> a/solr/core/src/test/org/apache/solr/update/processor/UpdateRequestProcessorFactoryTest.java
> b/solr/core/src/test/org/apache/solr/update/processor/UpdateRequestProcessorFactoryTest.java
> index 7d53b4f..66d612f 100644
> ---
> a/solr/core/src/test/org/apache/solr/update/processor/UpdateRequestProcessorFactoryTest.java
> +++
> b/solr/core/src/test/org/apache/solr/update/processor/UpdateRequestProcessorFactoryTest.java
> @@ -16,21 +16,21 @@
>   */
>  package org.apache.solr.update.processor;
>
> -import static
> org.apache.solr.update.processor.DistributingUpdateProcessorFactory.DISTRIB_UPDATE_PARAM;
> -
>  import java.lang.invoke.MethodHandles;
> -import java.util.Arrays;
>  import java.util.ArrayList;
> +import java.util.Arrays;
>  import java.util.List;
>
> +import org.apache.solr.SolrTestCaseJ4;
>  import org.apache.solr.common.params.ModifiableSolrParams;
>  import org.apache.solr.core.SolrCore;
>  import org.apache.solr.response.SolrQueryResponse;
> -import org.apache.solr.SolrTestCaseJ4;
>  import org.junit.BeforeClass;
>  import org.slf4j.Logger;
>  import org.slf4j.LoggerFactory;
>
> +import static
> org.apache.solr.update.processor.DistributingUpdateProcessorFactory.DISTRIB_UPDATE_PARAM;
> +
>  /**
>   *
>   */
> @@ -87,7 +87,7 @@ public class UpdateRequestProcessorFactoryTest extends
> SolrTestCaseJ4 {
>  assertEquals( custom, core.getUpdateProcessingChain( "custom" ) );
>
>  // Make sure the NamedListArgs got through ok
> -assertEquals( "{name={n8=88,n9=99}}", link.args.toString() );
> +assertEquals( "{name={n8=88, n9=99}}", link.args.toString() );
>}
>
>public void testUpdateDistribChainSkipping() throws Exception {
>
>

Re: [VOTE] Release Lucene/Solr 8.3.0 RC2

[David Smiley]

+1

SUCCESS! [1:25:00.693515]

Re: CVE-2018-11768 in regards to Solr

[Kevin Risden]

Solr interacts with Hadoop only as a client. (except in integration tests).
>From the https://hadoop.apache.org/cve_list.html page, this looks like it
is only an issue in the fsimage which is server side for the Namenode. I
don't think that CVE-2018-11768 applies to Solr directly.

CVE-2018-11768 Apache Hadoop HDFS FSImage Corruption
There is a mismatch in the size of the fields used to store user/group
information between memory and disk representation. This causes the
user/group information to be corrupted across storing in fsimage and
reading back from fsimage.

This vulnerability fix contains a fsimage layout change, so once the image
is saved in the new layout format you cannot go back to a version that
doesn’t support the newer layout. This means that once 2.7.x users upgraded
to the fixed version, they cannot downgrade to 2.7.x because there is no
fixed version in 2.7.x. We suggest downgrade to 2.8.5 or upper version that
contains the vulnerability fix.

Kevin Risden


On Tue, Oct 29, 2019 at 12:52 PM Kyle Gerald Lamkin 
wrote:

> Hello Solr Devs,
>
>
>
> I'm looking for some information about CVE-2018-11768, a Hadoop
> vulnerability. In 7.7.2 Solr ships with Hadoop 2.7.4 which is affected, the
> closest fixed version is 2.8.5. Solr 8.x ships with Hadoop 3.2 which is not
> affected.
>
>
>
> I was unable to find an Jira item for this, should I open one for it or is
> it not applicable?
>
>
>
> Thanks for your time.
>
>
> Regards,
>
> *Kyle (K.G.) Lamkin*
> --
> E-mail: kyle.lam...@ibm.com
>
>
> - To
> unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org

CVE-2018-11768 in regards to Solr

[Kyle Gerald Lamkin]

Hello Solr Devs,
 
I'm looking for some information about CVE-2018-11768, a Hadoop vulnerability. In 7.7.2 Solr ships with Hadoop 2.7.4 which is affected, the closest fixed version is 2.8.5. Solr 8.x ships with Hadoop 3.2 which is not affected.
 
I was unable to find an Jira item for this, should I open one for it or is it not applicable?
 
Thanks for your time.
 
 
Regards, 

Kyle (K.G.) Lamkin
E-mail: kyle.lam...@ibm.com
 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Rethinking how we publish the Solr Ref Guide

[Gézapeti]

Thanks for the help! The link was an internal documentation I assumed it
was correct. I've updated it in our wiki page.


On Mon, Oct 28, 2019 at 7:57 PM Cassandra Targett 
wrote:

> Yes, it does need to be updated. I was waiting to do that until I informed
> the user list about the change to not publish a PDF any longer which I’m
> ready to send now, so I’ll also fix the redirect link.
>
> Cassandra
> On Oct 28, 2019, 12:23 PM -0500, Gus Heck , wrote:
>
> Ah yes I assumed that the original link had come from a good source...
> OTOH
> https://lucene.apache.org/solr/guide/field-types-included-with-solr.html still
> needs to be updated to point to 8_2 I think.
>
> On Mon, Oct 28, 2019 at 1:01 PM Chris Hostetter 
> wrote:
>
>>
>> : The redirection is wrong, if you remove "latest" from the urls with 8_1
>> in
>>
>> The "redirection" rules appear to be working as designed -- but AFAIK they
>> were never designed with any idea of having a "latest/" path.
>>
>> the Latest URL has no is just the page name w/o a version number, not the
>> page name prefixed with "latest/".
>>
>> Specifically this is suppose to be the "latest" URL for the page in
>> question...
>>
>> https://lucene.apache.org/solr/guide/field-types-included-with-solr.html
>>
>> : > I was trying to access
>> : >
>> https://lucene.apache.org/solr/guide/latest/field-types-included-with-solr.html
>>
>> ...where did you find that link?
>>
>> I'm pretty sure the place that linked to that URL is the place that has a
>> mistake.
>>
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
>

Re: [VOTE] Release Lucene/Solr 8.3.0 RC2

[Adrien Grand]

+1 SUCCESS! [2:14:14.583794]

On Mon, Oct 28, 2019 at 9:44 PM Jitendra soni  wrote:

> I'm able to run it locally and did some verification.
>
> so it's  +1
>
> SUCCESS! [1:17:33.483295]
>
> [image: image.png]
>
> On Sat, Oct 26, 2019 at 5:32 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Please vote for release candidate 2 for Lucene/Solr 8.3.0
>>
>> The artifacts can be downloaded from:
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.3.0-RC2-rev2aa586909b911e66e1d8863aa89f173d69f86cd2
>>
>> You can run the smoke tester directly with this command:
>>
>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.3.0-RC2-rev2aa586909b911e66e1d8863aa89f173d69f86cd2
>>
>> The vote will be open for at least 3 working days, i.e. until
>> 2019-10-30 19:00 UTC.
>>
>> [*] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>>
>> Here is my +1
>> 
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>
> --
> Thanks
> Jitendra
>


-- 
Adrien

Re: Which is the most convenient technique to convert NSF to PST file?

[Alexandre Rafalovitch]

This is spam, right? Designed to show in the web archives. We should -
at least - block the user.

On the other hand, if somebody was actually interested in extracting
information from Lotus Notes, I do have an open-source tool for that.
https://github.com/arafalov/Lotus-Notes-Exporter

On Tue, 29 Oct 2019 at 21:29, kellyjohnson1  wrote:
>
>  As per the suggestions and experience of the majority of Lotus Notes users,
> the most convenient technique to  convert NSF to PST


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Which is the most convenient technique to convert NSF to PST file?

[kellyjohnson1]

 As per the suggestions and experience of the majority of Lotus Notes users,
the most convenient technique to  convert NSF to PST
   file type is by
using a trustworthy tool. One such type of application is developed by
*eSoftTools *and is known as the NSF Converter Tool. This software is
developed in such a manner that even a person who does not have technical
knowledge about Lotus Notes can operate it very efficiently. This software
has the ability to restore deleted email with all attributes like bcc, cc,
subject, hyperlinks, to, from, attachments, embedded images, and many
others. Even a huge PST file can be split into small packets of new PST
files. A free demo version is also offered to all users with support to all
editions of Lotus Notes and Outlook.
Explore more:  https://www.esofttools.com/nsf-to-pst-converter.html
  
 



--
Sent from: 
https://lucene.472066.n3.nabble.com/Lucene-Java-Developer-f564358.html

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org