Источник:
https://www.elastic.co/guide/en/elasticsearch/reference/current/targz.html
Установка:
$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.11.0-linux-x86_64.tar.gz $ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.11.0-linux-x86_64.tar.gz.sha512 $ shasum -a 512 -c elasticsearch-8.11.0-linux-x86_64.tar.gz.sha512 $ tar -xzf elasticsearch-8.11.0-linux-x86_64.tar.gz $ cd elasticsearch-8.11.0/
Перед запуском проверить JAVA_HOME:
$ echo $JAVA_HOME /usr/lib/jvm/java-1.11.0-openjdk-amd64/
Но elastic использует СВОЮ java JVM в "elasticsearch-8.1.1/jdk". Версия Java выводится в логе при запуске:
.... Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.2/17.0.2+8. .... "warning: ignoring JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64/; using bundled JDK") ....
Проверка версии Java в дистрибутиве Elastic:
@v:~/po/elasticsearch-8.1.1/jdk/bin$ ./java --version openjdk 17.0.2 2022-01-18
Запуск
@v:~/po/elasticsearch-8.1.1/bin$ ./elasticsearch
Log:
warning: ignoring JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64/; using bundled JDK
warning: ignoring JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64/; using bundled JDK
warning: ignoring JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64/; using bundled JDK
[INFO ][o.e.n.Node ] [v.perm.ru] version[8.1.1], pid[358443],
build[default/tar/d0925dd6f22e07b935750420a3155db6e5c58381/2022-03-17T22:01:32.658689558Z],
OS[Linux/6.8.0-47-generic/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.2/17.0.2+8]
[INFO ][o.e.n.Node ] [v.perm.ru] JVM home [/home/vasi/po/elasticsearch-8.1.1/jdk],
using bundled JDK [true]
[INFO ][o.e.n.Node ] [v.perm.ru] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60,
-Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m,
-Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow,
-XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true,
-Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0,
-Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true,
-Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC,
-Djava.io.tmpdir=/tmp/elasticsearch-6176186819399335581, -XX:+HeapDumpOnOutOfMemoryError,
-XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log,
-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m,
-Xms5822m, -Xmx5822m, -XX:MaxDirectMemorySize=3053453312, -XX:G1HeapRegionSize=4m,
-XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15,
-Des.path.home=/home/vasi/po/elasticsearch-8.1.1, -Des.path.conf=/home/vasi/po/elasticsearch-8.1.1/config,
-Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [aggs-matrix-stats]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [analysis-common]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [constant-keyword]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [data-streams]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [frozen-indices]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [ingest-common]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [ingest-geoip]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [ingest-user-agent]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [kibana]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [lang-expression]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [lang-mustache]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [lang-painless]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [legacy-geo]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [mapper-extras]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [mapper-version]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [old-lucene-versions]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [parent-join]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [percolator]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [rank-eval]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [reindex]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [repositories-metering-api]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [repository-azure]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [repository-encrypted]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [repository-gcs]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [repository-s3]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [repository-url]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [runtime-fields-common]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [search-business-rules]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [searchable-snapshots]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [snapshot-based-recoveries]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [snapshot-repo-test-kit]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [spatial]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [transform]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [transport-netty4]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [unsigned-long]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [vector-tile]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [vectors]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [wildcard]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-aggregate-metric]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-analytics]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-async]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-async-search]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-autoscaling]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-ccr]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-core]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-deprecation]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-enrich]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-eql]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-fleet]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-graph]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-identity-provider]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-ilm]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-logstash]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-ml]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-monitoring]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-ql]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-rollup]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-security]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-shutdown]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-sql]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-stack]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-text-structure]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-voting-only-node]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] loaded module [x-pack-watcher]
[INFO ][o.e.p.PluginsService ] [v.perm.ru] no plugins loaded
[INFO ][o.e.e.NodeEnvironment ] [v.perm.ru] using [1] data paths, mounts [[/ (/dev/sda3)]],
net usable_space [187.7gb], net total_space [365.2gb], types [ext4]
[INFO ][o.e.e.NodeEnvironment ] [v.perm.ru] heap size [5.6gb], compressed ordinary object pointers [true]
[INFO ][o.e.n.Node ] [v.perm.ru] node name [v.perm.ru], node ID [9NVvBuA6Qim3F5-_1CivHw],
cluster name [elasticsearch], roles [data_warm, data_content, transform, data_hot, ml, data_frozen,
ingest, data_cold, data, remote_cluster_client, master]
[INFO ][o.e.x.m.p.l.CppLogMessageHandler] [v.perm.ru] [controller/358722] [Main.cc@123]
controller (64 bit): Version 8.1.1 (Build f5fbdbeecb0643) Copyright (c) 2022 Elasticsearch BV
[INFO ][o.e.x.s.Security ] [v.perm.ru] Security is disabled
[INFO ][o.e.t.n.NettyAllocator ] [v.perm.ru] creating NettyAllocator with the following configs:
[name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb,
factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[INFO ][o.e.i.r.RecoverySettings ] [v.perm.ru] using rate limit [40mb] with
[default=40mb, read=0b, write=0b, max=0b]
[INFO ][o.e.d.DiscoveryModule ] [v.perm.ru] using discovery type [multi-node] and seed hosts providers [settings]
[INFO ][o.e.n.Node ] [v.perm.ru] initialized
[INFO ][o.e.n.Node ] [v.perm.ru] starting ...
[INFO ][o.e.x.s.c.f.PersistentCache] [v.perm.ru] persistent cache index loaded
[INFO ][o.e.x.d.l.DeprecationIndexingComponent] [v.perm.ru] deprecation component started
[INFO ][o.e.t.TransportService ] [v.perm.ru] publish_address {192.168.1.20:9300}, bound_addresses {192.168.1.20:9300}
[INFO ][o.e.b.BootstrapChecks ] [v.perm.ru] bound or publishing to a non-loopback address, enforcing bootstrap checks
[INFO ][o.e.c.c.Coordinator ] [v.perm.ru] cluster UUID [g3ojVaUHRuefR6zTbSa-FA]
[INFO ][o.e.c.s.MasterService ] [v.perm.ru] elected-as-master ([1] nodes joined)[{v.perm.ru}
{9NVvBuA6Qim3F5-_1CivHw}{6fEAYJNeR1WxjP2OmgsGOg}{192.168.1.20}{192.168.1.20:9300}{cdfhilmrstw} completing election,
_BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 12, version: 1ta: master node changed {previous [],
current [{v.perm.ru}{9NVvBuA6Qim3F5-_1CivHw}{6fEAYJNeR1WxjP2OmgsGOg}{192.168.1.20}{192.168.1.20:9300}{cdfhilmrstw}]}
[INFO ][o.e.c.s.ClusterApplierService] [v.perm.ru] master node changed {previous [], current
[{v.perm.ru}{9NVvBuA6Qim3F5-_1CivHw}{6fEAYJNeR1WxjP2OmgsGOg}{192.168.1.20}{192.168.1.20:9300}{cdfhilmrstw}]},
term: 12, version: 1son: Publication{term=12, version=159}
[INFO ][o.e.h.AbstractHttpServerTransport] [v.perm.ru] publish_address {192.168.1.20:9200},
bound_addresses {[::1]:9200}, {127.0.0.1:9200}, {192.168.1.20:9200}
[INFO ][o.e.n.Node ] [v.perm.ru] started
[INFO ][o.e.l.LicenseService ] [v.perm.ru] license [fc40e350-7072-41dc-a1c0-16b7a52c83e4] mode [basic] - valid
[INFO ][o.e.g.GatewayService ] [v.perm.ru] recovered [2] indices into cluster_state
[INFO ][o.e.c.r.a.AllocationService] [v.perm.ru] current.health="GREEN" message="Cluster health status changed
from [RED] to [GREEN] (reason: [shards started [[.geoip_databases][0]]])." previous.health="RED" reason="shards started [[.geoip_databases][0]]"
[INFO ][o.e.i.g.DatabaseNodeService] [v.perm.ru] retrieve geoip database [GeoLite2-City.mmdb] from [.geoip_databases]
to [/tmp/elasticsearch-6176186819399335581/geoip-databases/9NVvBuA6Qim3F5-_1CivHw/GeoLite2-City.mmdb.tmp.gz]
[INFO ][o.e.i.g.DatabaseNodeService] [v.perm.ru] retrieve geoip database [GeoLite2-Country.mmdb] from [.geoip_databases]
to [/tmp/elasticsearch-6176186819399335581/geoip-databases/9NVvBuA6Qim3F5-_1CivHw/GeoLite2-Country.mmdb.tmp.gz]
[INFO ][o.e.i.g.DatabaseNodeService] [v.perm.ru] retrieve geoip database [GeoLite2-ASN.mmdb] from [.geoip_databases]
to [/tmp/elasticsearch-6176186819399335581/geoip-databases/9NVvBuA6Qim3F5-_1CivHw/GeoLite2-ASN.mmdb.tmp.gz]
[INFO ][o.e.i.g.DatabaseNodeService] [v.perm.ru] successfully loaded geoip database file [GeoLite2-Country.mmdb]
[INFO ][o.e.i.g.DatabaseNodeService] [v.perm.ru] successfully loaded geoip database file [GeoLite2-ASN.mmdb]
[INFO ][o.e.i.g.DatabaseNodeService] [v.perm.ru] successfully loaded geoip database file [GeoLite2-City.mmdb]
----------------------------------------
publish_address {192.168.1.20:9300}, bound_addresses {192.168.1.20:9300}
[INFO ][o.e.l.LicenseService ] [v.perm.ru] license [fc40e350-7072-41dc-a1c0-16b7a52c83e4] mode [basic] - valid
Проверка:
vasi@v:~$ http http://127.0.0.1:9200
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-encoding: gzip
content-length: 324
content-type: application/json
{
"cluster_name": "elasticsearch",
"cluster_uuid": "g3ojVaUHRuefR6zTbSa-FA",
"name": "v.perm.ru",
"tagline": "You Know, for Search",
"version": {
"build_date": "2022-03-17T22:01:32.658689558Z",
"build_flavor": "default",
"build_hash": "d0925dd6f22e07b935750420a3155db6e5c58381",
"build_snapshot": false,
"build_type": "tar",
"lucene_version": "9.0.0",
"minimum_index_compatibility_version": "7.0.0",
"minimum_wire_compatibility_version": "7.17.0",
"number": "8.1.1"
}
}
С удаленного компьютера тоже работает:
vasi@vasi-note:~$ http http://192.168.1.20:9200
HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-encoding: gzip
content-length: 324
content-type: application/json
{
"cluster_name": "elasticsearch",
"cluster_uuid": "g3ojVaUHRuefR6zTbSa-FA",
"name": "v.perm.ru",
"tagline": "You Know, for Search",
"version": {
"build_date": "2022-03-17T22:01:32.658689558Z",
"build_flavor": "default",
"build_hash": "d0925dd6f22e07b935750420a3155db6e5c58381",
"build_snapshot": false,
"build_type": "tar",
"lucene_version": "9.0.0",
"minimum_index_compatibility_version": "7.0.0",
"minimum_wire_compatibility_version": "7.17.0",
"number": "8.1.1"
}
}
Состояние сервиса:
curl -X GET "http://192.168.1.20:9200/_cluster/health?pretty"
{
"cluster_name" : "elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 13,
"active_shards" : 13,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
Запуск как демон
Памяти сожррет около 6GB!
$ ./bin/elasticsearch -d -p pid
в файл "pid" запишется номер процеса.
Для остановки выполнить:
$ pkill -F pid
Основная конфигурация в elasticsearch-8.1.1/config/elasticsearch.yml (архив каталога config в https://github.com/cherepakhin/senior/blob/main/new_site/elasticsearch/config.zip)
Запуск как сервис в Linux
Содержимое файла /etc/systemd/system/elasticsearch.service:
[Unit] Description=Elasticsearch service After=network.target [Service] User=vasi Group=vasi Type=simple WorkingDirectory=/home/vasi/po/elasticsearch-8.1.1/ RuntimeDirectoryMode=0750 ExecStart=/home/vasi/po/elasticsearch-8.1.1/bin/elasticsearch -p pid User=vasi Restart=on-abort [Install] WantedBy=multi-user.target
Перечитать сервисы:
$ sudo systemctl daemon-reload
Старт сервиса:
$ systemctl start elasticsearch.service
Включить сервис в автозагрузку:
$ systemctl enable elasticsearch.service
Настройка elasticsearch.yml:
~/po/elasticsearch-8.1.1/config$ cat elasticsearch.yml .... # ---------------------------------- Network ----------------------------------- # # By default Elasticsearch is only accessible on localhost. Set a different # address here to expose this node on the network: # network.host: 192.168.1.20 # # By default Elasticsearch listens for HTTP traffic on the first free port it # finds starting at 9200. Set a specific HTTP port here: # http.port: 9200 # Enable security features xpack.security.enabled: false xpack.security.enrollment.enabled: false # Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents xpack.security.http.ssl: enabled: false keystore.path: certs/http.p12 # Enable encryption and mutual authentication between cluster nodes xpack.security.transport.ssl: enabled: false verification_mode: certificate keystore.path: certs/transport.p12 truststore.path: certs/transport.p12 # Create a new cluster with the current node only # Additional nodes can still join the cluster later cluster.initial_master_nodes: ["v.perm.ru"] # Allow HTTP API connections from localhost and local networks # Connections are encrypted and require user authentication http.host: [_local_, _site_]
Настройка выделения памяти делается в /home/vasi/po/elasticsearch-8.1.1/config/jvm.options:
.... ################################################################ ## IMPORTANT: JVM heap size ################################################################.... -Xms2g -Xmx2g ....
Остальные параметры по умолчанию.
Ссылки:
Run Elasticsearch locally
Изучаем ELK. Часть I — Установка Elasticsearch (Habr)
Изучаем ELK. Часть II — Установка Kibana и Logstash (Habr)
Изучаем ELK. Часть III — Безопасность (Habr)
Передатчик из kafka в Elasticsearch
Install Elasticsearch with Docker
Autostart сервиса в linux
Docker image ElasticSearch
Дистрибутивы Kibana
Дистрибутивы ElasticSearch