Latest

How to Install and Configure Elasticsearch Cluster with Multiple Nodes

In our previous elasticsearch tutorial, we mentioned how to install and setup a stand-alone elasticsearch occasion.

Whereas stand-alone set up is sweet for dev/check, for production, it is strongly recommended to setup elasticsearch cluster. Elasticsearch cluster has many benefits over stand-alone.

Advantages of Elastisearch Cluster

  1. Distributed knowledge: In cluster knowledge is distributed, replicated to different server. So In case of failure of 1 node, knowledge could be restored from duplicate node. It avoids single level of failure.
  2. Devoted Node roles: Every node has dedicated position assigned to it, that ensures particular position and position based mostly load distribution hence growing performance. Listed here are two essential node roles
  3. Knowledge node: These nodes solely shops knowledge and do knowledge associated operations, search and knowledge manipulation.
  4. Grasp node: Grasp of all nodes, it holds duty of general cluster, addition and removing of nodes from cluster, retaining monitor of alive nodes, grasp reselection in applicable instances.
  5. Scalability: Cluster model is definitely scalable to multiple no of nodes. Thus growing efficiency and reliability of elasticsearch.

On this tutorial, we’ll setup three node elasticsearch cluster.

Node 1 – Install and Configure

Let’s start putting in elasticsearch, I will present each and each steps throughout set up process

Comply with our previous article How to set up Elasticsearch and obtain elasticsearch zip archive, extract it on server or set up it utilizing deb repository.

Now open the Elasticsearch configuration file in extracted folder, or /and so on/elasticsearch in case of put in immediately from repo.

vim /and so on/elasticsearch/elasticsearch.yml

Now we could have to configure it as cluster nodes, following entries want to be modified in the above file.

Set the cluster identify:

cluster-name=TGS

Set the first node identify:

node.identify=tgs-1

Set whether this node will act as a grasp:

node.master=true

Set whether or not this node will act as a data-node:

node.knowledge=true

Set first node’s ip and port:

community.host=192.168.101.51
http.port=9200

Cluster nodes discovery protocol and record of nodes:

discovery.zen.ping.unicast.hosts: [“192.168.101.51”, “192.168.101.52”,”192.168.101.53″]

These are all of the nodes part of cluster, this makes node discovery straightforward.

Now begin elasticsearch server, following are essential a part of installation logs:

[INFO ][o.e.c.s.ClusterApplierService] [tgs-1] new_master tgs-17IqQvmfdSb66AFcP0u0AZQOveP8d_SR0WIjVupZcW35g192.168.101.51192.168.101.51:9300ml.machine_memory=134986862592xpack.installed=trueml.max_open_jobs=20ml.enabled=truecause: apply cluster state (from grasp [master tgs-17IqQvmfdSb66AFcP0u0AZQOveP8d_SR0WIjVupZcW35g192.168.101.51192.168.101.51:9300ml.machine_memory=134986862592xpack.installed=trueml.max_open_jobs=20ml.enabled=true committed version [1] supply [zen-disco-elected-as-master ([0] nodes joined)]])
[INFO ][o.e.h.n.Netty4HttpServerTransport] [tgs-1] publish_address 192.168.101.51:9200bound_addresses 192.168.101.51:9200
[INFO ][o.e.n.Node ] [tgs-1] began
[WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [tgs-1] Failed to clear cache for realms [[]] [INFO ][o.e.l.LicenseService ] [tgs-1] license [77777777-dddd-4444-aaaa-888888888888] mode [basic] – legitimate
[INFO ][o.e.g.GatewayService ] [tgs-1] recovered [0] indices into cluster_state
[INFO ][o.e.c.s.MasterService ] [tgs-1] zen-disco-node-join[tgs-2S64iPm03Qtq0-bvJipk-_AeAChESS6TceLHHTHyfMU7g192.168.101.52192.168.101.52:9300ml.machine_memory=134986862592ml.max_open_jobs=20xpack.installed=trueml.enabled=true]cause: added tgs-2S64iPm03Qtq0-bvJipk-_AeAChESS6TceLHHTHyfMU7g192.168.101.52192.168.101.52:9300ml.machine_memory=134986862592ml.max_open_jobs=20xpack.installed=trueml.enabled=true,
[INFO ][o.e.c.s.ClusterApplierService] [tgs-1] added tgs-2S64iPm03Qtq0-bvJipk-_AeAChESS6TceLHHTHyfMU7g192.168.101.52192.168.101.52:9300ml.machine_memory=134986862592ml.max_open_jobs=20xpack.put in=trueml.enabled=true,purpose: apply cluster state (from grasp [master tgs-17IqQvmfdSb66AFcP0u0AZQOveP8d_SR0WIjVupZcW35g192.168.101.51192.168.101.51:9300ml.machine_memory=134986862592xpack.installed=trueml.max_open_jobs=20ml.enabled=true committed version [4] supply[zen-disco-node-join[zen-disco-node-join[zen-disco-node-join[zen-disco-node-join[tgs-2S64iPm03Qtq0-bvJipk-_AeAChESS6TceLHHTHyfMU7g192.168.101.52192.168.101.52:9300ml.machine_memory=134986862592ml.max_open_jobs=20xpack.installed=trueml.enabled=true]]])

Node 2 – Install and Configure

login to second server, set up elasticsearch or extract from downloaded zip archive

Edit configuration file accordingly.

Set the cluster identify:

cluster-name=TGS

Set the second node identify:

node.identify=tgs-2

Set whether or not this node will act as a master:

node.grasp=true

Set whether or not this node will act as a data-node:

node.knowledge=true

Set second node’s ip and port:

network.host=192.168.101.52
http.port=9200

Cluster nodes discovery protocol and record of nodes:

discovery.zen.ping.unicast.hosts: [“192.168.101.51”, “192.168.101.52”,”192.168.101.53″]

These are all of the nodes part of cluster, this makes node discovery straightforward.

Now start elasticsearch server, following are essential a part of installation logs

[INFO ][o.e.c.s.ClusterApplierService] [tgs-2] detected_master tgs-1
[WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [tgs-2] Failed to clear cache for realms [[]] [INFO ][o.e.x.s.a.TokenService ] [tgs-2] refresh keys
[INFO ][o.e.x.s.a.TokenService ] [tgs-2] refreshed keys
[INFO ][o.e.l.LicenseService ] [tgs-2] license [77777777-dddd-4444-aaaa-888888888888] mode [basic] – valid
[INFO ][o.e.h.n.Netty4HttpServerTransport] [tgs-2] publish_address 192.168.101.52:9200, bound_addresses 192.168.101.52:9200
[INFO ][o.e.n.Node ] [tgs-2] began
[INFO ][o.e.c.s.ClusterApplierService] [tgs-2] added tgs-3

Node 3 – Install and Configure

Login to third server and install elasticsearch or extract downloaded zip archive

Set the cluster identify:

cluster-name=TGS

Set the third node identify:

node.identify=tgs-3

Set whether or not this node will act as a grasp:

node.master=true

Set whether this node will act as a data-node:

node.knowledge=true

Set second node’s ip and port:

network.host=192.168.101.53
http.port=9200

Cluster nodes discovery protocol and record of nodes

discovery.zen.ping.unicast.hosts: [“192.168.101.51”, “192.168.101.52”,”192.168.101.53″]

These are all of the nodes a part of cluster, this makes node discovery straightforward.

Now begin elasticsearch server, following are necessary part of set up logs

[INFO ][o.e.c.s.ClusterApplierService] [tgs-3] detected_master tgs-1
[WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [tgs-3] Failed to clear cache for realms [[]] [INFO ][o.e.x.s.a.TokenService ] [tgs-3] refresh keys
[INFO ][o.e.x.s.a.TokenService ] [tgs-3] refreshed keys
[INFO ][o.e.l.LicenseService ] [tgs-3] license [77777777-dddd-4444-aaaa-888888888888] mode [basic] – legitimate
[INFO ][o.e.h.n.Netty4HttpServerTransport] [tgs-3] publish_address 192.168.101.53:9200, bound_addresses 192.168.101.53:9200
[INFO ][o.e.n.Node ] [tgs-3] began

Confirm Elasticsearch Cluster Status

Our cluster is setup and operating, let’s confirm this cluster setup, we might use elastic search Cluster API to verify status of cluster (will cover intimately about cluster APIs in ES collection articles)

curl -XGET ‘http://localhost:9200/_cluster/state?pretty’

Output:

“cluster_name” : “TGS”,
“cluster_uuid” : “vlt_bBfJRP268MFFA5XY4w”,
“version” : 5,
“state_uuid” : “VxN1E51xRievom5n1BDiNQ”,
“master_node” : “7IqQvmfdSb66AFcP0u0AZQ”,
“blocks” : ,
“nodes” :
“S64iPm03Qtq0-bvJipk-_A” :
“name” : “tgs-2”,
“ephemeral_id” : “eAChESS6TceLHHTHyfMU7g”,
“transport_address” : “192.168.101.52:9300”,
“attributes” :
“ml.machine_memory” : “134986862592”,
“ml.max_open_jobs” : “20”,
“xpack.installed” : “true”,
“ml.enabled” : “true”

,
“syxQbgRPS4q992NXkx3ptw” :
“name” : “tgs-3”,
“ephemeral_id” : “-uK6TjQ2Qz6o_6arfmRR3Q”,
“transport_address” : “192.168.101.53:9300”,
“attributes” :
“ml.machine_memory” : “134986862592”,
“ml.max_open_jobs” : “20”,
“xpack.installed” : “true”,
“ml.enabled” : “true”

,
“7IqQvmfdSb66AFcP0u0AZQ” :
“name” : “tgs-1”,
“ephemeral_id” : “OveP8d_SR0WIjVupZcW35g”,
“transport_address” : “192.168.101.51:9300”,
“attributes” :
“ml.machine_memory” : “134986862592”,
“xpack.installed” : “true”,
“ml.max_open_jobs” : “20”,
“ml.enabled” : “true”

,

That is the significant a part of standing API, it exhibits cluster, nodes, transport tackle and ids.

curl -XGET ‘http://localhost:9200/_cluster/health’

Output:

“cluster_name” : “TGS”,
“status” : “green”,
“timed_out” : false,
“number_of_nodes” : Three,
“number_of_data_nodes” : 3,
“active_primary_shards” : zero,
“active_shards” : zero,
“relocating_shards” : 0,
“initializing_shards” : zero,
“unassigned_shards” : 0,
“delayed_unassigned_shards” : 0,
“number_of_pending_tasks” : 0,
“number_of_in_flight_fetch” : zero,
“task_max_waiting_in_queue_millis” : zero,
“active_shards_percent_as_number” : 100.zero

The above output shows the general health of the cluster and it’s present status is Green.

In case you enjoyed this article, you may also like..