25.03.06. ELK stack, 파일 압축과 해제

HAproxy → “LB”, reverse-proxy

Nginx → 1. Web 2. LB 3. Reverse-Proxy + CA(인증서 자체 발행 기관

sudo dnf -y install haproxy

# 실행파일
[user1@rocky1 ~]$ which haproxy
/usr/sbin/haproxy

# 구성파일
[user1@rocky1 ~]$ cat /etc/haproxy/haproxy.cfg
global  # HAProxy 시스템에 대한 설정값
   log /dev/log local0
   log /dev/log local1 notice
   chroot /var/lib/haproxy
   stats timeout 30s
   user haproxy
   group haproxy
   daemon

defaults
   log global  # 로그정보는 global 을 따른다
   mode http              # tcp 를 사용하는 로드밸런싱 모드
   option httplog        # 기본 로그는 SIP, DIP
   option dontlognull    # 스캔등에 대한 정보는 로그화하지 않음
   timeout connect 5s  # 백엔드 서버로 연결 시도 시 최대 5초 대기
   timeout client 1m  # 클라이언트가 1분 동안 아무런 데이터도 보내지 않으면 연결 종료
   timeout server 1m  # 서버가 1분 동안 응답하지 않으면 연결 종료

frontend http_front
   bind *:80
   stats uri /haproxy?stats  #<http://lb주소/haproxy?stats> 에서 확인
   default_backend http_back

backend http_back
   balance source       # hash -> source IP # source -> 전체 해시 재계산
   hash-type consistent # 문제 있는 서버로 연결된 세션만 가장 가까운 서버로 이동시키고 기존 열결에 문제가 없는 세션은 그대로 유
#   balance roundrobin
   server rocky2.crys.pri 211.183.3.102:80 check
   server rocky3.crys.pri 211.183.3.103:80 check

[user1@rocky1 ~]$ sudo systemctl start haproxy
[user1@rocky1 ~]$ sudo systemctl status haproxy
● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled; vendor preset: disabl>
   Active: active (running) since Thu 2025-03-06 09:38:52 KST; 6s ago
  Process: 2475 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -f $CFGDIR -c -q $OPTIONS (code=e>
 Main PID: 2478 (haproxy)
    Tasks: 2 (limit: 22872)
   Memory: 3.9M
   CGroup: /system.slice/haproxy.service
           ├─2478 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -f /etc/haproxy/conf.d >
           └─2480 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -f /etc/haproxy/conf.d >

Mar 06 09:38:52 rocky1.crys.pri systemd[1]: Starting HAProxy Load Balancer...
Mar 06 09:38:52 rocky1.crys.pri haproxy[2478]: Proxy http_front started.
Mar 06 09:38:52 rocky1.crys.pri haproxy[2478]: Proxy http_back started.

• rocky1에서 elasticsearch 설치

# elk 를 rocky 에서 설치하고자 하는 경우 기본 저장소 목록에서는 이를 제공하지 않는다. 따라서 elk 설치를 위한 저장소를 추가해야 한다. 
[user1@rocky1 ~]$ sudo rpm --import <https://artifacts.elastic.co/GPG-KEY-elasticsearch>
cat <https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

[user1@rocky1 ~]$ sudo cat /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

# elasticsearch 7.10.1 설치
[user1@rocky1 ~]$ sudo dnf install -y elasticsearch-7.10.1

# elasticsearch에 대한 구성
[user1@rocky1 ~]$ sudo vi /etc/elasticsearch/elasticsearch.yml
55   network.host: 0.0.0.0     # 55행 정도에 주석처리되어 있으면 수정
89   discovery.type: single-node   # 없는 내용이므로 가장 아래 줄에 추가

# elasticsearch 실행하기/상태확인
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch

• rocky1에서 kibana 설치

# kibana 7.10.1 설치
[user1@rocky1 ~]$ sudo dnf install -y kibana-7.10.1

# kibana에 대한 구성
[user1@rocky1 ~]$ sudo vi /etc/kibana/kibana.yml
7 server.host: "0.0.0.0"
28 elasticsearch.hosts: ["<http://localhost:9200>"]

# kibana 실행하기/상태확인
sudo systemctl start kibana
sudo systemctl status kibana

RDBMS ES

table	index → 여러 개가 묶이면 index patterns
row	document {key:value}

• rocky3(데이터 소스 → metric[CPU/RAM] 수집)에서 metricbeat설치
  • beats → metricbeat, filebeat, …

# 설치
sudo rpm --import <https://artifacts.elastic.co/GPG-KEY-elasticsearch>
cat <https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
sudo dnf install -y metricbeat-7.10.1

# 설정 변경
sudo vi /etc/metricbeat/metricbeat.yml
94 hosts: ["211.183.3.101:9200"]
67 host: ""

[user1@rocky3 ~]$ sudo ls /etc/metricbeat/modules.d/
[sudo] password for user1:
activemq.yml.disabled             golang.yml.disabled          nats.yml.disabled
aerospike.yml.disabled            googlecloud.yml.disabled     **nginx**.yml.disabled
apache.yml.disabled               graphite.yml.disabled        openmetrics.yml.disabled
appsearch.yml.disabled            **haproxy**.yml.disabled         oracle.yml.disabled
aws.yml.disabled                  http.yml.disabled            php_fpm.yml.disabled
azure.yml.disabled                ibmmq.yml.disabled           **postgresql**.yml.disabled
beat-xpack.yml.disabled           iis.yml.disabled             **prometheus**.yml.disabled
beat.yml.disabled                 istio.yml.disabled           rabbitmq.yml.disabled
ceph-mgr.yml.disabled             jolokia.yml.disabled         redisenterprise.yml.disabled
ceph.yml.disabled                 kafka.yml.disabled           redis.yml.disabled
cloudfoundry.yml.disabled         kibana-xpack.yml.disabled    sql.yml.disabled
cockroachdb.yml.disabled          kibana.yml.disabled          stan.yml.disabled
consul.yml.disabled               kubernetes.yml.disabled      statsd.yml.disabled
coredns.yml.disabled              kvm.yml.disabled             system.yml
couchbase.yml.disabled            linux.yml.disabled           tomcat.yml.disabled
couchdb.yml.disabled              logstash-xpack.yml.disabled  traefik.yml.disabled
**docker**.yml.disabled               logstash.yml.disabled        uwsgi.yml.disabled
dropwizard.yml.disabled           memcached.yml.disabled       vsphere.yml.disabled
elasticsearch-xpack.yml.disabled  mongodb.yml.disabled         windows.yml.disabled
elasticsearch.yml.disabled        mssql.yml.disabled           zookeeper.yml.disabled
envoyproxy.yml.disabled           munin.yml.disabled
etcd.yml.disabled                 **mysql**.yml.disabled
  
[user1@rocky3 ~]$ sudo cat /etc/metricbeat/modules.d/system.yml
# Module: system
# Docs: <https://www.elastic.co/guide/en/beats/metricbeat/7.10/metricbeat-module-system.html>

- module: system
  period: 10s
  metricsets:
    - cpu
    - load
    - memory
    - network
    - process
    - process_summary
    - socket_summary
    #- entropy
    #- core
    #- diskio
    #- socket
    #- service
    #- users
  process.include_top_n:
    by_cpu: 5      # include top 5 processes by CPU
    by_memory: 5   # include top 5 processes by memory

- module: system
  period: 1m
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)'

- module: system
  period: 15m
  metricsets:
    - uptime

#- module: system
#  period: 5m
#  metricsets:
#    - raid
#  raid.mount_point: '/'

[user1@rocky3 ~]$ sudo metricbeat modules enable linux
Enabled linux

파일 압축과 묶기

리눅스/유닉스에서는 원칙적으로 ‘파일 압축’과 ‘파일 묶기’를 별개의 프로그램으로 처리

[user1@rocky1 ~]$ mkdir test
[user1@rocky1 ~]$ touch test/{a,b,c}.txt
[user1@rocky1 ~]$ ls test/
a.txt  b.txt  c.txt

[user1@rocky1 ~]$ tar cvfJ test.tar.xz test/ # 묶고 압축
test/
test/a.txt
test/b.txt
test/c.txt
[user1@rocky1 ~]$ ls
Desktop    Downloads  mytestkey.pem      Pictures  Templates  **test.tar.xz**
Documents  Music      mytestkey.pem.pub  Public    test       Videos
[user1@rocky1 ~]$ tar xfJ test.tar.xz # 압축 해제 및 풀기

# 변수 설정 및 적용(로그처럼 백업을 위해 사용)
[user1@rocky1 ~]$ thistime=$(date +%y%m%d%H%M) ; tar cfJ $thistime.tar.xz test
[user1@rocky1 ~]$ ls
2503061230.tar.xz  Desktop    Downloads  mytestkey.pem      Pictures  Templates  Videos
date               Documents  Music      mytestkey.pem.pub  Public    test

CRON, AT

• cron- 주기적으로 반복되는 시스템 작업을 예약
• at- 일회성 시스템 작업을 예약

분 시 일 월 요일 사용자 실행명령