Skip to content

Elastic stack

You know, for search

source: How we cooked Elasticsearch, Consul, HAproxy and DNS-recursor

Basics

An Elasticsearch index is based on Lucene indices. An index can be sharded on multiple nodes.

The searching process is as follows:

  • ES serach index 1
  • Lucene search index 1.1
  • Lucene search index 1.2
  • Lucene search index 1.N
  • ES merge search results

A search query returns documents. Documents can have types. So, an index can store multiple document types.

A document exposes multiple fields. Fields can be reused across different document types. Fields have types, too (ex: keyword type)

In the design phase, you must choose between storing multiple document types in 1 index, or storing them in multiple indices.

source: Elastic.co - index vs type

Mapping

A mapping defines how the data is stored and indexed in Elasticsearch. A common problem with mappings is the mappings explosion, which sounds fun.

source: Elastic.co - Reference: Mapping

Using Logstash

with Logstash Elasticsearch output plugin in outputs:

1
2
3
4
5
6
7
output {
  elasticsearch {
    hosts => [ "127.0.0.1" ]
    index => "my_specific_index"
    document_type => "mytype"
  }
}

When sending data to ES with this output conf, it auto-generates a mapping (=Dynamic mapping, ES is actually guessing types and sizes).

Display the mapping with: GET my_specific_index/_mapping?pretty

Mapping is a nested structure. A properties block encapsulates each level of the structure.

Creating a custom mapping (Explicit mapping) is important because ES' guesses are whack (ie: using double where smaller values could to the job, like half_floats, or typing IP adresses as string).

sources: elastic.co - Logstash lesson Elasticsearch mapping pt2

Templates

Format json