Package org.eclipse.rdf4j.sail.elasticsearch

Class ElasticsearchIndex

java.lang.Object

org.eclipse.rdf4j.sail.lucene.AbstractSearchIndex

org.eclipse.rdf4j.sail.elasticsearch.ElasticsearchIndex

All Implemented Interfaces:

SearchIndex

public class ElasticsearchIndex
extends AbstractSearchIndex

Requires an Elasticsearch cluster with the DeleteByQuery plugin. Note that, while RDF4J is licensed under the EDL, several ElasticSearch dependencies are licensed under the Elastic license or the SSPL, which may have implications for some projects. Please consult the ElasticSearch website and license FAQ for more information.

See Also:

Elastic License FAQ, LuceneSail

Field Summary

Fields

Modifier and Type	Field	Description
static String	ALL_PROPERTY_FIELDS
static String	DEFAULT_ANALYZER
static String	DEFAULT_DOCUMENT_TYPE
static String	DEFAULT_INDEX_NAME
static String	DEFAULT_TRANSPORT
static String	DOCUMENT_TYPE_KEY	Set the parameter "documentType=" to specify the document type to use.
static String	ELASTICSEARCH_KEY_PREFIX
static String	GEOPOINT_FIELD_PREFIX
static String	GEOSHAPE_FIELD_PREFIX
static String	INDEX_NAME_KEY	Set the parameter "indexName=" to specify the index to use.
static String	PROPERTY_FIELD_PREFIX
static String	TRANSPORT_KEY	Set the parameter "transport=" to specify the address of the cluster to use (e.g.
static String	WAIT_FOR_ACTIVE_SHARDS_KEY	Set the parameter "waitForActiveShards=" to configure if initialization should wait until the specified number of shards to be active.
static String	WAIT_FOR_NO_RELOCATING_SHARDS_KEY	Set the parameter "waitForNoRelocatingShards=true|false" to configure if initialization should wait until the are no relocating shards.
static String	WAIT_FOR_NODES_KEY	Set the parameter "waitForNodes=" to configure if initialization should wait until the specified number of nodes are available.
static String	WAIT_FOR_RELOCATING_SHARDS_KEY	Deprecated. use WAIT_FOR_NO_RELOCATING_SHARDS_KEY in elastic search >= 5.x
static String	WAIT_FOR_STATUS_KEY	Set the parameter "waitForStatus=" to configure if initialization should wait for a particular health status.

Fields inherited from class org.eclipse.rdf4j.sail.lucene.AbstractSearchIndex

maxDocs, wktFields

Constructor Summary

Constructors

Constructor	Description
ElasticsearchIndex()

Method Summary

Modifier and Type	Method	Description
protected void	addDocument(SearchDocument doc)
void	begin()	Begins a transaction.
void	clear()	Clears the indexes.
void	clearContexts(Resource... contexts)	This should be called from within a begin-commit-rollback block.
void	commit()	Commits any changes done to the LuceneIndex since the last commit.The semantics is synchronous to SailConnection.commit(), i.e.
protected SearchDocument	copyDocument(SearchDocument doc)
protected com.google.common.base.Function<? super String,? extends org.locationtech.spatial4j.context.SpatialContext>	createSpatialContextMapper(Map<String,String> parameters)
protected void	deleteDocument(SearchDocument doc)
protected Iterable<? extends DocumentDistance>	geoQuery(IRI geoProperty, org.locationtech.spatial4j.shape.Point p, IRI units, double distance, String distanceVar, Var contextVar)
protected Iterable<? extends DocumentResult>	geoRelationQuery(String relation, IRI geoProperty, String wkt, Var contextVar)
String	getClusterName()
protected SearchDocument	getDocument(String id)	Returns a Document representing the specified document ID (combination of resource and context), or null when no such Document exists yet.
SearchDocument	getDocument(Resource subject, Resource context)	Returns a Document representing the specified Resource and Context combination, or null when no such Document exists yet.
protected Iterable<? extends SearchDocument>	getDocuments(String resourceId)
Iterable<? extends SearchDocument>	getDocuments(Resource subject)	Returns a list of Documents representing the specified Resource (empty when no such Document exists yet).Each document represent a set of statements with the specified Resource as a subject, which are stored in a specific context
String	getIndexName()
Map<String,Object>	getMappings()
static Set<String>	getPropertyFields(Set<String> fields)	Filters the given list of fields, retaining all property fields.
protected org.locationtech.spatial4j.context.SpatialContext	getSpatialContext(String property)
String[]	getTypes()
void	initialize(Properties parameters)
protected BulkUpdater	newBulkUpdate()
protected SearchDocument	newDocument(String id, String resourceId, String context)
protected Iterable<? extends DocumentScore>	query(Resource subject, QuerySpec spec)	Parse the passed query.
void	rollback()
org.elasticsearch.search.SearchHits	search(Resource resource, org.elasticsearch.action.search.SearchRequestBuilder request, org.elasticsearch.index.query.QueryBuilder query)	Evaluates the given query only for the given resource.
org.elasticsearch.search.SearchHits	search(org.elasticsearch.action.search.SearchRequestBuilder request, org.elasticsearch.index.query.QueryBuilder query)	Evaluates the given query and returns the results as a TopDocs instance.
void	shutDown()
protected void	updateDocument(SearchDocument doc)

Methods inherited from class org.eclipse.rdf4j.sail.lucene.AbstractSearchIndex

accept, addDocuments, addRemoveStatements, addStatement, evaluate, getIndexedTypeMapping, getResource, isGeoField, isIndexedTypeStatement, isTypeFilteringEnabled, isTypeStatement, parseLuceneQueryShape, parseQueryPoint, parseQueryShape, removeStatement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INDEX_NAME_KEY

public static final String INDEX_NAME_KEY

Set the parameter "indexName=" to specify the index to use.

See Also:

Constant Field Values

DOCUMENT_TYPE_KEY

public static final String DOCUMENT_TYPE_KEY

Set the parameter "documentType=" to specify the document type to use. By default, the document type is "resource".

See Also:

Constant Field Values

TRANSPORT_KEY

public static final String TRANSPORT_KEY

Set the parameter "transport=" to specify the address of the cluster to use (e.g. localhost:9300).

See Also:

Constant Field Values

WAIT_FOR_STATUS_KEY

public static final String WAIT_FOR_STATUS_KEY

Set the parameter "waitForStatus=" to configure if initialization should wait for a particular health status. The value can be one of "green" or "yellow". Does not wait by default.

See Also:

Constant Field Values

WAIT_FOR_NODES_KEY

public static final String WAIT_FOR_NODES_KEY

Set the parameter "waitForNodes=" to configure if initialization should wait until the specified number of nodes are available. Does not wait by default.

See Also:

Constant Field Values

WAIT_FOR_ACTIVE_SHARDS_KEY

public static final String WAIT_FOR_ACTIVE_SHARDS_KEY

Set the parameter "waitForActiveShards=" to configure if initialization should wait until the specified number of shards to be active. Does not wait by default.

See Also:

Constant Field Values

WAIT_FOR_RELOCATING_SHARDS_KEY

@Deprecated
public static final String WAIT_FOR_RELOCATING_SHARDS_KEY

Deprecated.

use WAIT_FOR_NO_RELOCATING_SHARDS_KEY in elastic search >= 5.x

Set the parameter "waitForRelocatingShards=" to configure if initialization should wait until the specified number of nodes are relocating. Does not wait by default.

See Also:

Constant Field Values

WAIT_FOR_NO_RELOCATING_SHARDS_KEY

public static final String WAIT_FOR_NO_RELOCATING_SHARDS_KEY

Set the parameter "waitForNoRelocatingShards=true|false" to configure if initialization should wait until the are no relocating shards. Defaults to false, meaning the operation does not wait on there being no more relocating shards. Set to true to wait until the number of relocating shards in the cluster is 0.

See Also:

Constant Field Values

DEFAULT_INDEX_NAME

public static final String DEFAULT_INDEX_NAME

See Also:

Constant Field Values

DEFAULT_DOCUMENT_TYPE

public static final String DEFAULT_DOCUMENT_TYPE

See Also:

Constant Field Values

DEFAULT_TRANSPORT

public static final String DEFAULT_TRANSPORT

See Also:

Constant Field Values

DEFAULT_ANALYZER

public static final String DEFAULT_ANALYZER

See Also:

Constant Field Values

ELASTICSEARCH_KEY_PREFIX

public static final String ELASTICSEARCH_KEY_PREFIX

See Also:

Constant Field Values

PROPERTY_FIELD_PREFIX

public static final String PROPERTY_FIELD_PREFIX

See Also:

Constant Field Values

ALL_PROPERTY_FIELDS

public static final String ALL_PROPERTY_FIELDS

See Also:

Constant Field Values

GEOPOINT_FIELD_PREFIX

public static final String GEOPOINT_FIELD_PREFIX

See Also:

Constant Field Values

GEOSHAPE_FIELD_PREFIX

public static final String GEOSHAPE_FIELD_PREFIX

See Also:

Constant Field Values

Constructor Detail

ElasticsearchIndex

public ElasticsearchIndex()

Method Detail

getClusterName

public String getClusterName()

getIndexName

public String getIndexName()

getTypes

public String[] getTypes()

initialize

public void initialize(Properties parameters)
                throws Exception

Specified by:

initialize in interface SearchIndex

Overrides:

initialize in class AbstractSearchIndex

Throws:

Exception

createSpatialContextMapper

protected com.google.common.base.Function<? super String,? extends org.locationtech.spatial4j.context.SpatialContext> createSpatialContextMapper(Map<String,String> parameters)

getMappings

public Map<String,Object> getMappings()
                                     throws IOException

Throws:

IOException

getSpatialContext

protected org.locationtech.spatial4j.context.SpatialContext getSpatialContext(String property)

Specified by:

getSpatialContext in class AbstractSearchIndex

shutDown

public void shutDown()
              throws IOException

Throws:

IOException

getDocument

protected SearchDocument getDocument(String id)
                              throws IOException

Returns a Document representing the specified document ID (combination of resource and context), or null when no such Document exists yet.

Specified by:

getDocument in class AbstractSearchIndex

Throws:

IOException

getDocuments

protected Iterable<? extends SearchDocument> getDocuments(String resourceId)
                                                   throws IOException

Specified by:

getDocuments in class AbstractSearchIndex

Throws:

IOException

newDocument

protected SearchDocument newDocument(String id,
                                     String resourceId,
                                     String context)

Specified by:

newDocument in class AbstractSearchIndex

copyDocument

protected SearchDocument copyDocument(SearchDocument doc)

Specified by:

copyDocument in class AbstractSearchIndex

addDocument

protected void addDocument(SearchDocument doc)
                    throws IOException

Specified by:

addDocument in class AbstractSearchIndex

Throws:

IOException

updateDocument

protected void updateDocument(SearchDocument doc)
                       throws IOException

Specified by:

updateDocument in class AbstractSearchIndex

Throws:

IOException

deleteDocument

protected void deleteDocument(SearchDocument doc)
                       throws IOException

Specified by:

deleteDocument in class AbstractSearchIndex

Throws:

IOException

newBulkUpdate

protected BulkUpdater newBulkUpdate()

Specified by:

newBulkUpdate in class AbstractSearchIndex

getDocument

public SearchDocument getDocument(Resource subject,
                                  Resource context)
                           throws IOException

Returns a Document representing the specified Resource and Context combination, or null when no such Document exists yet.

Parameters:

subject -

context -

Returns:

search document

Throws:

IOException

getDocuments

public Iterable<? extends SearchDocument> getDocuments(Resource subject)
                                                throws IOException

Returns a list of Documents representing the specified Resource (empty when no such Document exists yet).Each document represent a set of statements with the specified Resource as a subject, which are stored in a specific context

Parameters:

subject -

Returns:

list of documents

Throws:

IOException

getPropertyFields

public static Set<String> getPropertyFields(Set<String> fields)

Filters the given list of fields, retaining all property fields.

Parameters:

fields -

Returns:

set of fields

begin

public void begin()
           throws IOException

Description copied from interface: SearchIndex

Begins a transaction.

Throws:

IOException

commit

public void commit()
            throws IOException

Description copied from interface: SearchIndex

Commits any changes done to the LuceneIndex since the last commit.The semantics is synchronous to SailConnection.commit(), i.e. the LuceneIndex should be committed/rolled back whenever the LuceneSailConnection is committed/rolled back.

Throws:

IOException

rollback

public void rollback()
              throws IOException

Throws:

IOException

query

protected Iterable<? extends DocumentScore> query(Resource subject,
                                                  QuerySpec spec)
                                           throws MalformedQueryException,
                                                  IOException

Parse the passed query.

Specified by:

query in class AbstractSearchIndex

Parameters:

subject -

spec - query to process

Returns:

the parsed query

Throws:

MalformedQueryException

IOException

IllegalArgumentException - if the spec contains a multi-param query

search

public org.elasticsearch.search.SearchHits search(Resource resource,
                                                  org.elasticsearch.action.search.SearchRequestBuilder request,
                                                  org.elasticsearch.index.query.QueryBuilder query)

Evaluates the given query only for the given resource.

Parameters:

resource -

request -

query -

Returns:

search hits

geoQuery

protected Iterable<? extends DocumentDistance> geoQuery(IRI geoProperty,
                                                        org.locationtech.spatial4j.shape.Point p,
                                                        IRI units,
                                                        double distance,
                                                        String distanceVar,
                                                        Var contextVar)
                                                 throws MalformedQueryException,
                                                        IOException

Specified by:

geoQuery in class AbstractSearchIndex

Throws:

MalformedQueryException

IOException

geoRelationQuery

protected Iterable<? extends DocumentResult> geoRelationQuery(String relation,
                                                              IRI geoProperty,
                                                              String wkt,
                                                              Var contextVar)
                                                       throws MalformedQueryException,
                                                              IOException

Specified by:

geoRelationQuery in class AbstractSearchIndex

Throws:

MalformedQueryException

IOException

search

public org.elasticsearch.search.SearchHits search(org.elasticsearch.action.search.SearchRequestBuilder request,
                                                  org.elasticsearch.index.query.QueryBuilder query)

Evaluates the given query and returns the results as a TopDocs instance.

clearContexts

public void clearContexts(Resource... contexts)
                   throws IOException

Description copied from interface: SearchIndex

This should be called from within a begin-commit-rollback block.

Parameters:

contexts -

Throws:

IOException

clear

public void clear()
           throws IOException

Description copied from interface: SearchIndex

Clears the indexes.

Throws:

IOException