Full-text search does not work

richardk · November 20, 2015, 11:41am

Hello, after creating my first journal and uploading also the articles in pdf I did a test searching words but the full text or “search all” does not find the pdf content, what do I need to do to solve that?

ctgraham · November 20, 2015, 1:17pm

Review the settings in search section of config.inc.php.

github.com

pkp/ojs/blob/ojs-dev-2_4/config.TEMPLATE.inc.php#L346-L376



[search]

; Minimum indexed word length
min_word_length = 3

; The maximum number of search results fetched per keyword. These results
; are fetched and merged to provide results for searches with several keywords.
results_per_keyword = 500

; The number of hours for which keyword search results are cached.
result_cache_hours = 1

; Paths to helper programs for indexing non-text files.
; Programs are assumed to output the converted text to stdout, and "%s" is
; replaced by the file argument.
; Note that using full paths to the binaries is recommended.
; Uncomment applicable lines to enable (at most one per file type).
; Additional "index[MIME_TYPE]" lines can be added for any mime type to be
; indexed.

This file has been truncated. show original

Particularly, do you have a command uncommented for each filetype (application/pdf, application/postscript, msword) that you want to search inside? Does the command execute if run by the web user?

richardk · January 20, 2017, 3:04pm

Dear Alec, I made sure that the settings you provided was reviewed, nevertheless the search is still not working. If I enter a word like “Auctores” which is the title of a journal and which is also appearing in the Journal page, in the Index page and in the PDF, no results are found. Since some months ago it was working…
Can you please help us to reactivate the full-text search please?
Thank you very much and best regards,
Richard

asmecher · January 20, 2017, 5:17pm

Hi @richardk,

The OJS internal search engine does not index all aspects of the journal site – it indexes article content only. If you want a more general search feature, you might look into using a Google custom search engine.

Regards,
Alec Smecher
Public Knowledge Project Team

richardk · January 21, 2017, 5:26pm

Hello Alec,

I’m not looking for something particular, the basic search is just not working
actually.
As a proof, some months ago I did a screen shot of a search for “Taddeo”
receiving several search results

if one does the same search now it is answering “no results”, which means the
OJS erach is actually not working at all

Thank you for your help and best regards,
Richard

ctgraham · January 23, 2017, 4:03pm

I’d like to return to the earlier question: What commands are in use within config.inc.php for each filetype (application/pdf, application/postscript, msword) that you want to search inside? Does the command execute if run by the web user?

Also, have you tried the rebuildSearchIndex.php in the “tools” folder? What success or error output does it return? For example, if I were running it, I would run this as sudo su apache -s /bin/bash -c 'php tools/rebuildSearchIndex.php' (but your command line will vary).

richardk · January 23, 2017, 4:48pm

Dear Clinton,

thank you very much for your reply, I have forwarded to my colleagues your
tecnical informations and hope we can solve this problem asap.

thanks,
Richard

richardk · January 25, 2017, 6:11pm

Dear Clinton,
my technician reported me that launcing the search command php
rebuildSearchIndex.php

the following error shows up:

LucenePlugin: Clearing index … done
Indexing “AUCTORES NOSTRI” error: An error occurred while indexing: Processed
16 out of a batch of 28 (28 deleted documents in this batch).

The error on the indexing log is:
WARNING: Error creating document :
SolrInputDocument[{article_id=article_id(1.0)={30},
journal_id=journal_id(1.0)={1}, loadAction=loadAction(1.0)={delete},
etl_galley_xml=etl_galley_xml(1.0)={<?xml version=“1.0”
encoding=“utf-8”?>}, section_id=section_id(1.0)={},
inst_id=inst_id(1.0)={localhost}, etl_suppFile_xml=etl_suppFile_xml(1.0)={<?xml
version=“1.0” encoding=“utf-8”?>},
plainText=plainText(1.0)={30}}]
org.apache.solr.common.SolrException: ERROR: [doc=30] Error adding field
‘section_id’=’’
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:333)

Can these details be helpful?

I tried therefore to disable the Generic Plugin “Lucene Search Plugin” in the
Journal “Auctores Nostri” and repeated a search but it is still not working.
What does this message “processed 16 out of a batch of 28” mean? Is it possible
that some PDF file I uploaded for the journal “Auctores Nostri” is corrupted?

Thank you very much to help us further.

best regards,
Richard

ctgraham · January 25, 2017, 8:17pm

That certainly makes it sound like Solr (Lucene Search Plugin) can’t handle something midway through the process.

If the tech reruns rebuildSearchIndex.php while you have the Lucene Search Plugin disabled, does the process complete or error out?

richardk · January 26, 2017, 3:16pm

Hello,

I guess thanks to your help we found out a little bug in OJS
Effectively, after disabling Lucene and rerunning SearchIndex the full text
search returned working!

I will post this on OJS forum in case someone else experience the same problem.

Thanks and best regards,
Richard

richardk · January 26, 2017, 3:18pm

Great news!
effectively there was a problem with the “Lucene Search Plugin”.
After disabling it and rerunning “rebuildSearchIndex” the Full Text search returned to work perfectively
Thank you very much.

best regards,
Richard

Dulip_Withanage · March 8, 2022, 7:08am

Hi @richardk ,

As part of the optimization of external search support for OJS, the PKP Technical Committee is looking for use cases and relevant experience with the Lucene/Solr plugin for OJS.

GitHub - ojsde/lucene: Plugin for Solr/Lucene support in OJS. or via Plugin-Gallery

We are going to organize a meeting with all the interested parties. If you are interested in sharing requirements, user experiences and use cases, please feel free to contact me.

Regards,
Dulip Withanage