ElasticSearch集群搭建(三节点)
1.centos7安装了jdk1.8版本
2.下载elasticsearch6.8版本,下载地址为:elasticsearch-6-8-0" rel="nofollow">https://www.elastic.co/cn/downloads/past-releases/elasticsearch-6-8-0
3.上传到三台虚拟机上,创建/etc/software,将安装文件解压到该文件夹下
elasticsearchyml192168164019216816411921681642_4">4.进入安装目录,修改config文件夹下的elasticsearch.yml文件,修改内容如下(所用三台机器为192.168.16.40,192.168.16.41,192.168.16.42)
(1)40服务器参数配置:
# Use a descriptive name for the node:
#
cluster.name: test-master
node.name: test-node1
#
#设置充当master节点,默认为true
node.master: true
#
#设置不充当data节点,默认为true
node.data: false
network.host: 192.168.16.40
network.bind_host: 192.168.16.40
network.publish_host: 192.168.16.40
#
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.16.41:9300","192.168.16.42:9300"]
(2)41服务器参数配置:
# Use a descriptive name for the node:
#
cluster.name: test-master
node.name: test-node2
#
#设置充当master节点,默认为true
node.master: false
#设置不充当data节点,默认为true
node.data: true
#network.host: 192.168.0.1
network.host: 192.168.16.41
network.bind_host: 192.168.16.41
network.publish_host: 192.168.16.41
#
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.16.40:9300","192.168.16.42:9300"]
(3)42服务器参数配置:
#cluster.name: my-application
#
cluster.name: test-master
node.name: test-node3
#
#设置充当master节点,默认是true
node.master: false
#设置不充当data节点,默认为true
node.data: true
#network.host: 192.168.0.1
network.host: 192.168.16.42
network.bind_host: 192.168.16.42
network.publish_host: 192.168.16.42
#
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.16.40:9300","192.168.16.41:9300"]
elasticsearchrootelasticsearch_113">5.因为elasticsearch不支持root用户启动,所以创建一个普通用户来启动elasticsearch
Adduser elk
Passwd 123456
然后给elk用户授权:(在安装目录下)
chown -R elk:elk elasticsearch-6.8.0/
然后就可以切换elk用户启动elasticsearch
[root@slave01 bin]# su elk
[elk@slave01 bin]$ pwd
/etc/software/elasticsearch-6.8.0/bin
[elk@slave01 bin]$ ./elasticsearch
6.启动过程遇到的问题
1.问题一
[WARN ][o.e.b.JNANatives ] unable to install syscall filter:
Java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMPandCONFIG_SECCOMP_FILTERcompiledinatorg.elasticsearch.bootstrap.Seccomp.linuxImpl(Seccomp.java:349) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.bootstrap.Seccomp.init(Seccomp.java:630) ~[elasticsearch-5.0.0.jar:5.0.0]
原因:只是一个警告,主要是因为Linux版本过低造成的。
解决方案:
1、重新安装新版本的Linux系统
2、警告不影响使用,可以忽略
2.问题二
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
原因:无法创建本地文件问题,用户最大可创建文件数太小
解决方案:
切换到root用户,编辑limits.conf配置文件, 添加类似如下内容:
vi /etc/security/limits.conf
添加如下内容
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
备注:* 代表Linux所有用户名称(比如 hadoop)
保存、退出、重新登录才可生效
3.问题三
max number of threads [1024] for user [es] likely too low, increase to at least [2048]
原因:无法创建本地线程问题,用户最大可创建线程数太小
解决方案:切换到root用户,进入limits.d目录下,修改90-nproc.conf 配置文件
vi /etc/security/limits.d/90-nproc.conf
找到如下内容:
* soft nproc 1024
#修改为
* soft nproc 2048
4.问题四
max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
原因:最大虚拟内存太小
解决方案:切换到root用户下,修改配置文件sysctl.conf
vi /etc/sysctl.conf
添加下面配置:
vm.max_map_count=655360
并执行命令:
sysctl -p
然后重新启动elasticsearch,即可启动成功。
5.问题五
ElasticSearch启动找不到主机或路由
原因:ElasticSearch 单播配置有问题
解决方案:
检查ElasticSearch中的配置文件
vi config/elasticsearch.yml
找到如下配置:
discovery.zen.ping.unicast.hosts: ["172.16.31.220", "172.16.31.221","172.16.31.224"]
一般情况下,是这里配置有问题,注意书写格式
6.问题六
org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream
原因:ElasticSearch节点之间的jdk版本不一致
解决方案:ElasticSearch集群统一jdk环境
7.问题七
Failed to send join request to master [{node-1}{WbcP0pC_T32jWpYvu5is1A}{2_LCVHx1QEaBZYZ7XQEkMg}{10.10.11.200}{10.10.11.200:9300}],
reason [RemoteTransportException[[node-1][10.10.11.200:9300][internal:discovery/zen/join]];
nested: IllegalArgumentException[can't add node {node-2}{WbcP0pC_T32jWpYvu5is1A}{p-HCgFLvSFaTynjKSeqXyA}{10.10.11.200}{10.10.11.200:9301},
found existing node {node-1}{WbcP0pC_T32jWpYvu5is1A}{2_LCVHx1QEaBZYZ7XQEkMg}{10.10.11.200}{10.10.11.200:9300} with the same id but is a different node instance]; ]
问题原因:要是部署的时候从一个节点复制elasticsearch文件夹,其他节点可能包含被复制节点的data文件数据,需要把data文件下的文件清空