Re: Re: [EXTERNAL] Re: Apache Jena Full Text Search With Lucene

Lorenz Buehmann Wed, 28 Feb 2024 23:02:43 -0800

Hi,

just having the text index declared in the assembler doesn't trigger afull rebuild for obvious reasons. For SPARQL updates while runningFuseki indeed the fulltext index will be updated as it listens to anychanges in the underlying dataset.

In your case, you can run an offline indexing for your existing dataset,it is documented here:https://jena.apache.org/documentation/query/text-query.html#step-2---build-the-text-index


Hope this helps.


Cheers,
Lorenz

On 29.02.24 07:05, Chirag Ratra wrote:

Hi David,

Hurrah!! Query worked when I added new data and queried it. *Again BUT*
But I already have the dataset, I need to create an index on existing, how
can it be done?

Regards
Chirag

On Thu, Feb 29, 2024 at 11:25 AM David Habgood <[email protected]> wrote:

Hi Chirag,

How are you generating the database initially? You need to use the same
assembler file when doing this (alternately data you add now should be
added to the index - you can test this).

Thanks

On Thu, Feb 29, 2024 at 1:24 PM Chirag Ratra <[email protected]>
wrote:

Hi David,

I am able to query RDF data , but not FTS index

This query works :

PREFIX text: <http://jena.apache.org/text#>
PREFIX relation: <http://www.iana.org/assignments/relation/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX skosxl: <http://www.w3.org/2008/05/skos-xl#>
PREFIX relations: <https://cxdata.bold.com/ontologies/CareerDomain#>

select *  WHERE
{
?prefLabel skosxl:literalForm ?literalForm
   FILTER(?literalForm = 'Cashier')
}
limit 10


But this query doesn't

SELECT * WHERE {
  ?s text:query 'Cashier'
} LIMIT 10


Here is the ttl file


@prefix : <https://data.coypu.org/> .
@prefix fuseki: <http://jena.apache.org/fuseki#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix text: <http://jena.apache.org/text#> .
@prefix tdb2: <http://jena.apache.org/2016/tdb#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix skosxl: <http://www.w3.org/2008/05/skos-xl#> .

:service_my_ds rdf:type fuseki:Service;
     fuseki:name "tq-dataset";
     fuseki:endpoint [ fuseki:operation fuseki:query; fuseki:name "sparql"
];
     fuseki:endpoint [ fuseki:operation fuseki:query; fuseki:name "query"

];

     fuseki:endpoint [ fuseki:operation fuseki:update; fuseki:name

"update"

];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; ];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; fuseki:name "get" ];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-rw; fuseki:name "data"

];

     fuseki:dataset <#myTextDS> .

<#myTextDS> rdf:type text:TextDataset ;
     text:dataset <#myDatasetReadWrite> ;
     text:index <#indexLucene> ;
     .

<#indexLucene> a text:TextIndexLucene ;
     text:analyzer [ a text:StandardAnalyzer ];
     text:directory "run/databases/text-index-with-rdfs";
     text:storeValues true ;
     text:entityMap <#entMap> ;
     .

<#entMap> a text:EntityMap ;
     text:entityField "uri" ;
     text:graphField "graph" ;
     text:defaultField "prefLabel" ;
     text:map (
         [ text:field "prefLabel"; text:predicate skosxl:literalForm; ]
     ) .

<#myDatasetReadWrite> rdf:type tdb2:DatasetTDB2 ;
    tdb2:location

"C:\\Users\\chirag.ratra\\Downloads\\apache-jena-fuseki-4.10.0\\apache-jena-fuseki-4.10.0\\run/databases/tq-dataset"

.

On Tue, Feb 27, 2024 at 6:49 PM David Habgood <[email protected]>
wrote:

Hi Chirag,

There's an extra triple in the last block:

<#myDatasetReadWrite> rdf:type tdb2:DatasetTDB2 ;
                       tdb2:unionDefaultGraph true ;
    tdb2:location

"C:\\Users\\chirag.ratra\\Downloads\\apache-jena-fuseki-4.10.0\\apache-jena-fuseki-4.10.0\\run/databases/tq-dataset"

.

Try removing the unionDefaultGraph true statement, so the last block
becomes:

<#myDatasetReadWrite> rdf:type tdb2:DatasetTDB2 ;
    tdb2:location

"C:\\Users\\chirag.ratra\\Downloads\\apache-jena-fuseki-4.10.0\\apache-jena-fuseki-4.10.0\\run/databases/tq-dataset"

.
I'm guessing this is your issue but if not can you go to the info tab

in

the fuseki UI and tell us what count triples returns, and this SPARQL
query:
SELECT * { GRAPH <urn:x-arq:DefaultGraph> {?s ?p ?o} } LIMIT 10

Thanks

On Tue, Feb 27, 2024 at 9:54 PM Chirag Ratra <[email protected]>
wrote:

Hi David,

fuseski-server.jar ran successfully but I am not able to see any

data,

please let me know If am i missing something?
Since I had the configuration for my rdf dataset in tq-dataset.ttl

file

and

had fuseki:name "tq-dataset" and I was able to query for my rdf data

(it

had 300000 triples)

If I update  tq-dataset.ttl  with the content you shared, I am not

able

to

find any triple.

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT * WHERE {
   ?sub ?pred ?obj .
} LIMIT 10

Here is the ttl file


@prefix : <https://data.coypu.org/> .
@prefix fuseki: <http://jena.apache.org/fuseki#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix text: <http://jena.apache.org/text#> .
@prefix tdb2: <http://jena.apache.org/2016/tdb#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix skosxl: <http://www.w3.org/2008/05/skos-xl#> .

:service_my_ds rdf:type fuseki:Service;
     fuseki:name "tq-dataset";
     fuseki:endpoint [ fuseki:operation fuseki:query; fuseki:name

"sparql"

];
     fuseki:endpoint [ fuseki:operation fuseki:query; fuseki:name

"query"

];

     fuseki:endpoint [ fuseki:operation fuseki:update; fuseki:name

"update"

];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; ];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; fuseki:name

"get"

];

     fuseki:endpoint [ fuseki:operation fuseki:gsp-rw; fuseki:name

"data"

];

     fuseki:dataset <#myTextDS> .

<#myTextDS> rdf:type text:TextDataset ;
     text:dataset <#myDatasetReadWrite> ;
     text:index <#indexLucene> ;
     .

<#indexLucene> a text:TextIndexLucene ;
     text:analyzer [ a text:StandardAnalyzer ];
     text:directory "run/databases/text-index-with-rdfs";
     text:storeValues true ;
     text:entityMap <#entMap> ;
     .

<#entMap> a text:EntityMap ;
     text:entityField "uri" ;
     text:graphField "graph" ;
     text:defaultField "prefLabel" ;
     text:map (
         [ text:field "prefLabel"; text:predicate skosxl:literalForm;

         [ text:field "altLabel"; text:predicate skosxl:literalForm; ]
     ) .

<#myDatasetReadWrite> rdf:type tdb2:DatasetTDB2 ;
                       tdb2:unionDefaultGraph true ;
    tdb2:location

"C:\\Users\\chirag.ratra\\Downloads\\apache-jena-fuseki-4.10.0\\apache-jena-fuseki-4.10.0\\run/databases/tq-dataset"

.



On Tue, Feb 27, 2024 at 4:15 PM David Habgood <[email protected]>
wrote:

Hi Cherag,

This is the file - it works for me with Jena 4.9.0. I am testing

with

an

existing dataset I have, only changing skosxl -> skos.

@prefix : <https://data.coypu.org/> .
@prefix fuseki: <http://jena.apache.org/fuseki#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix text: <http://jena.apache.org/text#> .
@prefix tdb2: <http://jena.apache.org/2016/tdb#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix skosxl: <http://www.w3.org/2008/05/skos-xl#> .

:service_my_ds rdf:type fuseki:Service;
     fuseki:name "my-dataset";
     fuseki:endpoint [ fuseki:operation fuseki:query; fuseki:name

"sparql"

];
     fuseki:endpoint [ fuseki:operation fuseki:query; fuseki:name

"query"

];

     fuseki:endpoint [ fuseki:operation fuseki:update; fuseki:name

"update"

];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; ];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; fuseki:name

"get"

];

     fuseki:endpoint [ fuseki:operation fuseki:gsp-rw; fuseki:name

"data"

];

     fuseki:dataset <#myTextDS> .

<#myTextDS> rdf:type text:TextDataset ;
     text:dataset <#myDatasetReadWrite> ;
     text:index <#indexLucene> ;
     .

<#indexLucene> a text:TextIndexLucene ;
     text:analyzer [ a text:StandardAnalyzer ];
     text:directory
"databases/text-index-with-rdfs";
     text:storeValues true ;
     text:entityMap <#entMap> ;
     .

<#entMap> a text:EntityMap ;
     text:entityField "uri" ;
     text:graphField "graph" ;
     text:defaultField "prefLabel" ;
     text:map (
         [ text:field "prefLabel"; text:predicate skosxl:prefLabel;

         [ text:field "altLabel"; text:predicate skosxl:altLabel; ]
     ) .

<#myDatasetReadWrite> rdf:type tdb2:DatasetTDB2 ;
                       tdb2:unionDefaultGraph true ;
     tdb2:location "databases/fair-ease-with-rdfs"
;
     .

On Tue, Feb 27, 2024 at 6:25 PM Chirag Ratra <

[email protected]>

wrote:

Hi David,

Could you please share the complete file with updated typos,

since

when I

am trying I am facing the same error.

On Tue, Feb 27, 2024 at 12:10 PM David Habgood <

[email protected]

wrote:

I made a typo - the reference to `<#myDatasetReadWrite>`

(within

the

<#myTextDS> definition) should match the subject of the last

triples

block `:myDatasetReadWrite`.
Try changing one of these to match the other.

On Tue, Feb 27, 2024 at 4:23 PM Chirag Ratra <

[email protected]>

wrote:

Hi David,

*I am getting this error :*

11:48:25 INFO  Config          :: Load configuration:

file:///

path/run/configuration/tq-dataset.ttl
11:48:25 ERROR Server          :: Exception in

initialization:

Can't

find

<file:///path/run/configuration/tq-dataset.ttl#myDatasetReadWrite>

as a

subject
11:48:25 WARN  WebAppContext   :: Failed startup of context
o.e.j.w.WebAppContext@787e4357{Apache Jena Fuseki

Server,/,file:///C:/Users/chirag.ratra/Downloads/apache-jena-fuseki-4.10.0/apache-jena-fuseki-4.10.0/webapp/,UNAVAILABLE}

org.apache.jena.assembler.exceptions.AssemblerException:

Can't

find

<file:///C:/Users/chirag.ratra/Downloads/apache-jena-fuseki-4.10.0/apache-jena-fuseki-4.10.0/run/configuration/tq-dataset.ttl#myDatasetReadWrite>

as a subject
         at

org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.open(AssemblerGroup.java:137)

~[fuseki-server.jar:4.10.0]
         at

org.apache.jena.assembler.assemblers.AssemblerGroup$ExpandingAssemblerGroup.open(AssemblerGroup.java:93)

~[fuseki-server.jar:4.10.0]


*Here is content of my file*

@prefix : <https://data.coypu.org/> .
@prefix fuseki: <http://jena.apache.org/fuseki#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix text: <http://jena.apache.org/text#> .
@prefix tdb2: <http://jena.apache.org/2016/tdb#> .
@prefix skosxl: <http://www.w3.org/2008/05/skos-xl#> .

:service_my_ds rdf:type fuseki:Service;
     fuseki:name "my-dataset";
     fuseki:endpoint [ fuseki:operation fuseki:query;

fuseki:name

"sparql"

];
     fuseki:endpoint [ fuseki:operation fuseki:query;

fuseki:name

"query"

];

     fuseki:endpoint [ fuseki:operation fuseki:update;

fuseki:name

"update"

];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; ];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r;

fuseki:name

"get"

];

     fuseki:endpoint [ fuseki:operation fuseki:gsp-rw;

fuseki:name

"data"

];

     fuseki:dataset <#myTextDS> .

<#myTextDS> rdf:type text:TextDataset ;
     text:dataset <#myDatasetReadWrite> ;
     text:index <#indexLucene> ;
     .

<#indexLucene> a text:TextIndexLucene ;
     text:analyzer [ a text:StandardAnalyzer ];
     text:directory

"C:\\Users\\chirag.ratra\\Downloads\\apache-jena-fuseki-4.10.0\\apache-jena-fuseki-4.10.0\\run/databases";

     text:storeValues true ;
     text:entityMap <#entMap> ;
     .

<#entMap> a text:EntityMap ;
     text:entityField "uri" ;
     text:graphField "graph" ;
     text:defaultField "prefLabel" ;
     text:map (
         [ text:field "prefLabel"; text:predicate

skosxl:prefLabel;

         [ text:field "altLabel"; text:predicate

skosxl:altLabel;

     ) .

:myDatasetReadWrite rdf:type tdb2:DatasetTDB2 ;
     tdb2:location

"C:\\Users\\chirag.ratra\\Downloads\\apache-jena-fuseki-4.10.0\\apache-jena-fuseki-4.10.0\\run/databases/my-dataset"

;

On Tue, Feb 27, 2024 at 10:58 AM David Habgood <

[email protected]

wrote:

Hi Chirag,

Try this, it is based on a working config I use - I

inadvertently

removed

some of your endpoint definitions, you may want to add

these

back

in.

@prefix : <https://data.coypu.org/> .
@prefix fuseki: <http://jena.apache.org/fuseki#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

@prefix text: <http://jena.apache.org/text#> .
@prefix tdb2: <http://jena.apache.org/2016/tdb#> .
@prefix skosxl: <http://www.w3.org/2008/05/skos-xl#> .

:service_my_ds rdf:type fuseki:Service;
     fuseki:name "my-dataset";
     fuseki:endpoint [ fuseki:operation fuseki:query;

fuseki:name

"sparql"

];
     fuseki:endpoint [ fuseki:operation fuseki:query;

fuseki:name

"query"

];

     fuseki:endpoint [ fuseki:operation fuseki:update;

fuseki:name

"update"

];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r; ];
     fuseki:endpoint [ fuseki:operation fuseki:gsp-r;

fuseki:name

"get"

];

     fuseki:endpoint [ fuseki:operation fuseki:gsp-rw;

fuseki:name

"data"

];

     fuseki:dataset <#myTextDS> .

<#myTextDS> rdf:type text:TextDataset ;
     text:dataset <#myDatasetReadWrite> ;
     text:index <#indexLucene> ;
     .

<#indexLucene> a text:TextIndexLucene ;
     text:analyzer [ a text:StandardAnalyzer ];
     text:directory

"path\\run/databases/my-dataset-text-index";

     text:storeValues true ;
     text:entityMap <#entMap> ;
     .

<#entMap> a text:EntityMap ;
     text:entityField "uri" ;
     text:graphField "graph" ;
     text:defaultField "prefLabel" ;
     text:map (
         [ text:field "prefLabel"; text:predicate

skosxl:prefLabel;

         [ text:field "altLabel"; text:predicate

skosxl:altLabel;

     ) .

:myDatasetReadWrite rdf:type tdb2:DatasetTDB2 ;
     tdb2:location "path\\run/databases/my-dataset" ;
     .

On Tue, Feb 27, 2024 at 2:49 PM Chirag Ratra <

[email protected]>

wrote:

@prefix :       <http://base/#> .
@prefix fuseki: <http://jena.apache.org/fuseki#> .
@prefix ja:     <

http://jena.hpl.hp.com/2005/11/Assembler#

@prefix rdf:    <

http://www.w3.org/1999/02/22-rdf-syntax-ns#

@prefix rdfs:   <http://www.w3.org/2000/01/rdf-schema#>

@prefix tdb2:   <http://jena.apache.org/2016/tdb#> .

:service_tdb_all  rdf:type  fuseki:Service;
         rdfs:label       "TDB2 tq-dataset";
         fuseki:dataset   :tdb_dataset_readwrite;
         fuseki:endpoint  [ fuseki:name       "sparql";
                            fuseki:operation  fuseki:query
                          ];
         fuseki:endpoint  [ fuseki:name       "update";
                            fuseki:operation

fuseki:update

                          ];
         fuseki:endpoint  [ fuseki:operation

fuseki:gsp-rw

];

         fuseki:endpoint  [ fuseki:name       "query";
                            fuseki:operation  fuseki:query
                          ];
         fuseki:endpoint  [ fuseki:operation  fuseki:query

];

         fuseki:endpoint  [ fuseki:name       "data";
                            fuseki:operation

fuseki:gsp-rw

                          ];
         fuseki:endpoint  [ fuseki:name       "get";
                            fuseki:operation  fuseki:gsp-r
                          ];
         fuseki:endpoint  [ fuseki:operation

fuseki:update

];

         fuseki:name      "tq-dataset" .


:tdb_dataset_readwrite
         rdf:type       tdb2:DatasetTDB2;
         tdb2:location  "path\\run/databases/tq-dataset" .


On Tue, Feb 27, 2024 at 10:09 AM David Habgood <

[email protected]

wrote:

Hi Chirag,

The mailing list does not support attachments, assuming

your

assembler

file

isn't too long you can just copy it into the email

body.

Thanks

On Tue, Feb 27, 2024 at 2:25 PM Chirag Ratra <

[email protected]>

wrote:

Hi Andy,
I have subscribed  to the mailing list, here is my

query

I am having a problem setting up the full text search

with

Apache

Jena

Fuseki.
I have a dataset named tq-dataset (which has an

assembler

file

/run/configuration/tq-dataset.ttl), it has skos xl

preferred

labels

which

have literal forms.  I need to create a fulltext

search

index

on

the

literal form of skos xl-prefLabel and skos

xl-altLabel.

Through FTS documentation I am not getting an idea

what

changes

should

be

made and where and I am not sure what configuration

changes

to

be

done

in

which configuration file. I  have shared the

configuration

file

for

my

dataset.




On Mon, Feb 26, 2024 at 3:26 PM Andy Seaborne <

[email protected]

wrote:

Hi there,

You don't appear to be subscribed to the

[email protected]

mailing

list with this email address, or you sent a message

before

replying

to

the acknowledgement. Your email has ended up in the

moderators

inbox.

If you are not subscribed, you will not see replies

to

your

question.

Please could you subscribe by sending an email to

[email protected]

then reply to the confirmation.

After that, please could you re-send the message.

You will then receive all replies that are sent to

the

list.

More details:

http://jena.apache.org/help_and_support/index.html

     Andy

email address used: [email protected]

You probably want to show the configuration file you

have

tried

when

posting your question.


On 26/02/2024 08:30, Chirag Ratra wrote:

Hi ,

I am having a problem setting up the full text

search

with

Apache

Jena

Fuseki.
I have a dataset named tq-dataset (which has an

assembler

file

/run/configuration/tq-dataset.ttl), it has skos xl

preferred

labels

which

have literal forms.  I need to index these literal

forms.

But

through

documentation I am not sure what new configuration

or

modification

existing

ttl files has to be done.

Regards
Chirag



This email may contain material that is

confidential,

privileged,

or

for

the sole use of the intended recipient.  Any review,

disclosure,

reliance,

or distribution by others or forwarding without

express

permission

is

strictly prohibited.  If you are not the intended

recipient,

please

contact

the sender and delete all copies, including

attachments.

This email may contain material that is confidential,

privileged,

or

for

the sole use of the intended recipient.  Any review,

disclosure,

reliance,

or distribution by others or forwarding without

express

permission

is

strictly prohibited.  If you are not the intended

recipient,

please

contact

the sender and delete all copies, including

attachments.

--








This email may contain material that is confidential,

privileged,

or for the sole use of the intended recipient.  Any

review,

disclosure,

reliance, or distribution by others or forwarding without

express

permission is strictly prohibited.  If you are not the

intended

recipient,

please contact the sender and delete all copies,

including

attachments.

--








This email may contain material that is confidential,

privileged,

or for the sole use of the intended recipient.  Any review,

disclosure,

reliance, or distribution by others or forwarding without

express

permission is strictly prohibited.  If you are not the

intended

recipient,

please contact the sender and delete all copies, including

attachments.

--








This email may contain material that is confidential, privileged,
or for the sole use of the intended recipient.  Any review,

disclosure,

reliance, or distribution by others or forwarding without express
permission is strictly prohibited.  If you are not the intended

recipient,

please contact the sender and delete all copies, including

attachments.

--








This email may contain material that is confidential, privileged,
or for the sole use of the intended recipient.  Any review,

disclosure,

reliance, or distribution by others or forwarding without express
permission is strictly prohibited.  If you are not the intended

recipient,

please contact the sender and delete all copies, including

attachments.

--








This email may contain material that is confidential, privileged,
or for the sole use of the intended recipient.  Any review, disclosure,
reliance, or distribution by others or forwarding without express
permission is strictly prohibited.  If you are not the intended

recipient,

please contact the sender and delete all copies, including attachments.

--
Lorenz Bühmann
Research Associate/Scientific Developer

Email [email protected]

Institute for Applied Informatics e.V. (InfAI) | Goerdelerring 9 | 04109 
Leipzig | Germany

Re: Re: [EXTERNAL] Re: Apache Jena Full Text Search With Lucene

Reply via email to