Elastic stack
You know, for search
source: How we cooked Elasticsearch, Consul, HAproxy and DNS-recursor
Basics
An Elasticsearch index is based on Lucene indices. An index can be sharded on multiple nodes.
The searching process is as follows:
- ES serach index 1
- Lucene search index 1.1
- Lucene search index 1.2
- Lucene search index 1.N
- ES merge search results
A search query returns documents. Documents can have types. So, an index can store multiple document types.
A document exposes multiple fields. Fields can be reused across different document types. Fields have types, too (ex: keyword
type)
In the design phase, you must choose between storing multiple document types in 1 index, or storing them in multiple indices.
source: Elastic.co - index vs type
Mapping
A mapping defines how the data is stored and indexed in Elasticsearch. A common problem with mappings is the mappings explosion, which sounds fun.
source: Elastic.co - Reference: Mapping
Using Logstash
with Logstash Elasticsearch output plugin in outputs
:
1 2 3 4 5 6 7 | output { elasticsearch { hosts => [ "127.0.0.1" ] index => "my_specific_index" document_type => "mytype" } } |
When sending data to ES with this output conf, it auto-generates a mapping (=Dynamic mapping, ES is actually guessing types and sizes).
Display the mapping with: GET my_specific_index/_mapping?pretty
Mapping is a nested structure. A properties
block encapsulates each level of the structure.
Creating a custom mapping (Explicit mapping) is important because ES' guesses are whack (ie: using double
where smaller values could to the job, like half_floats
, or typing IP adresses as string
).
sources: elastic.co - Logstash lesson Elasticsearch mapping pt2
Templates
Format json