Elasticsearch跨叢集搜尋(Cross Cluster Search)
1、簡介
Elasticsearch在5.3版本中引入了Cross Cluster Search(CCS 跨叢集搜尋)功能,用來替換掉要被廢棄的Tribe Node。類似Tribe Node,Cross Cluster Search用來實現跨叢集的資料搜尋。
2、配置Cross Cluster Search
假設我們有2個ES叢集:
Node |
Address |
Port |
Transport Port |
Cluster |
elasticsearch01 |
127.0.0.1 |
9201 |
9301 |
America |
elasticsearch02 |
127.0.0.1 |
9202 |
9302 |
America |
elasticsearch03 |
127.0.0.1 |
9203 |
9303 |
Europe |
elasticsearch04 |
127.0.0.1 |
9204 |
9304 |
Europe |
有2種方式可以用來配置CCS:
1)配置elasticsearch.yml
search: remote: america: seeds: 127.0.0.1:9301 seeds: 127.0.0.1:9302 europe: seeds: 127.0.0.1:9303 seeds: 127.0.0.1:9304
注意:以上方式,在配置的時候,需要remote cluster處在執行狀態。比如在配置“america”的叢集的時候,需要“europe”叢集處在執行狀態,否則節點無法啟動成功。
2)使用ofollow,noindex" target="_blank">Cluster Settings API 配置
curl -XPUT -H'Content-Type: application/json' localhost:9201/_cluster/settings -d ' { "persistent": { "search.remote": { "america": { "skip_unavailable": "true", "seeds": ["127.0.0.1:9301","127.0.0.1:9302"] }, "europe": { "skip_unavailable": "true", "seeds": ["127.0.0.1:9303","127.0.0.1:9304"] } } } }'
推薦使用API方式,可以方便的修改remote cluster的seeds和其他配置。
3、驗證Cross Cluster Search
1)使用_remote/info檢視CCS連線狀態:
[root@localhost elasticsearch01]# curl -XGET -H 'Content-Type: application/json' localhost:9201/_remote/info?pretty { "america" : { "seeds" : [ "127.0.0.1:9301", "127.0.0.1:9302" ], "http_addresses" : [ "127.0.0.1:9201", "127.0.0.1:9202" ], "connected" : true, "num_nodes_connected" : 2, "max_connections_per_cluster" : 3, "initial_connect_timeout" : "30s" }, "europe" : { "seeds" : [ "127.0.0.1:9303", "127.0.0.1:9304" ], "http_addresses" : [ "127.0.0.1:9203", "127.0.0.1:9204" ], "connected" : true, "num_nodes_connected" : 2, "max_connections_per_cluster" : 3, "initial_connect_timeout" : "30s" } }
2)使用跨叢集搜尋:
同時查詢2個叢集的資料:
GET /cluster_name:index,cluster_name:index/_search GET */index/_search
java API 示例:
//查詢所有叢集,以appIndex-開頭的資料 SearchRequest searchRequest = Requests.searchRequest("*:appIndex-*"); SearchResponse response = es.getClient().search(searchRequest).get();
4、Disable Cross Cluster Search
使用API設定:
curl -XPUT -H'Content-Type: application/json' localhost:9201/_cluster/settings -d ' { "persistent": { "search.remote": { "america": { "skip_unavailable": null, "seeds": null }, "europe": { "skip_unavailable": null, "seeds": null } } } }'
5、CCS的配置
search.remote.${cluster_alias}.skip_unavailable:查詢的時候skip不可達的叢集,預設false,推薦設定為true
search.remote.connect:預設true,即任何node都作為一個cross-cluster client去連線remote cluster,跨叢集搜尋的請求必須發給cross-cluster client。
search.remote.node.attr:設定remote node的屬性,比如search.remote.node.attr:gateway這樣設定,只有node.attr.gateway: true的node才會被其他的cross-cluster client連線用來做CCS查詢。
參考:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-cross-cluster-search.html
http://kelonsoftware.com/elasticsearch-cross-cluster-search/