Elasticsearch日志收集

Install pip if necessary

curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"

python get-pip.py

Install Curator for Elasticsearch

Elasticsearch Curator helps you curate, or manage, your Elasticsearch indices and snapshots by:

Obtaining the full list of indices (or snapshots) from the cluster, as the actionable list
Iterate through a list of user-defined filters to progressively remove indices (or snapshots) from this actionable list as needed.
Perform various actions on the items which remain in the actionable list.

pip install elasticsearch-curator
pip install click==6.7

Configure curator

mkdir -p /var/log/elastic
touch /var/log/elastic/curator.log
mkdir ~/.curator
vi ~/.curator/curator.yml

curator.yml

# Remember, leave a key empty if there is no value. None will be a string,
## not a Python "NoneType"
client:
hosts: [Elasticsearch Server IP]
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
 
logging:
loglevel: INFO
logfile: /var/log/elastic/curator.log
logformat: default
blacklist: ['elasticsearch', 'urllib3']

Have a test, now you can get the indices list
curator_cli show_indices

Create repository

Configure elasticseach.yml default in /etc/elasticsearch/elasticsearch.yml

elasticsearch.yml

path.repo:  /u01/elasticsearch/backup
http.max_header_size: 16kb

Restart elasticsearch service (service elasticsearch restart) to make the configurations work.

Create repository elasticsearch. Ensure location points to a valid path which is configured in path.repo, accesable from all nodes.

curl -XPUT http://localhost:9200/_snapshot/es_backup -H "Content-Type: application/json" -d @repository.json

repository.json

{
   "type": "fs",
   "settings": {
      "compress": true,
      "location": "/u01/elasticsearch/backup"
   }
}

Have a test

curl -XGET 'localhost:9200/_snapshot/_all?pretty=true'

Create curator yaml action files

daily_backup.yml

Customize the snapshot name in name option
action 1: backup all indices before today to repository elasticsearch with specified snapshot name
action 2: delete indices older than 185 days

daily_backup.yml

---
actions:
  1:
    action: snapshot
    description: >-
      Snapshot selected all indices to repository 'elasticsearch' with the snapshot name
    options:
      repository: es_backup
      name: '<c4cert-{now/d-1d}>'
      wait_for_completion: True
      max_wait: 4800
      wait_interval: 30
    filters:
    - filtertype: age
      source: name
      direction: older
      unit: days
      unit_count: 1
      timestring: "%Y.%m.%d"
 
 
  2:
    action: delete_indices
    description: >-
      Delete indices which is older than 185 days
    filters:
    - filtertype: age
      source: name
      direction: older
      unit: days
      unit_count: 185
      timestring: "%Y.%m.%d"

del_snapshot.yml
action 1: Delete snapshots from repository elasticsearch which is older than 185 days

del_snapshot.yml

---
 
actions:
  1:
    action: delete_snapshots
    description: >-
      Delete snapshots from repository which is older than 185 days
    options:
      repository: es_backup
      retry_interval: 120
      retry_count: 3
    filters:
    - filtertype: age
      source: creation_date
      direction: older
      unit: days
      unit_count: 185

restore.yml
action 1: Restore all indices in the most recent snapshot with state SUCCESS.

restore.yml

---
 
actions:
  1:
    action: restore
    description: >-
      Restore all indices in the most recent snapshot with state SUCCESS.  Wait
      for the restore to complete before continuing.  Do not skip the repository
      filesystem access check.  Use the other options to define the index/shard
      settings for the restore.
    options:
      repository: es_backup
      # If name is blank, the most recent snapshot by age will be selected
      name:
      # If indices is blank, all indices in the snapshot will be restored
      indices:
      wait_for_completion: True
      max_wait: 3600
      wait_interval: 10
    filters:
    - filtertype: state
      state: SUCCESS

Note: use --dry-run option to verify your action without any change. Find the dry run results in log path.
Curator --dry-run daily_backup.yml

Shell script and crontab

run.sh

#!/bin/sh
curator /u01/curator/del_snapshot.yml
curator /u01/curator/daily_backup.yml

crontab -e

Here configured the job run on every 3 AM

crontab

0 3 * * * /bin/sh /u01/curator/run.sh

Restore

Curator restore.yml

Tested OK in CERT env.

Some useful API

# get all repositories
curl -XGET 'localhost:9200/_snapshot/_all?pretty=true'
 
# delete repository
curl -XDELETE 'localhost:9200/_snapshot/es-snapshot?pretty=true'
 
# show snapshots
curator_cli show_snapshots --repository es_backup
 
# show indices
curator_cli show_indices