Setup Haproxy Dan Keepalived Di Debian 10
Posted on
Untuk membuat HA HAProxy on premise, aku pakai Keepalived. Keepalived disini berperan sebagai penjaga Floating IP, untuk fail over ketika master0 mati maka keepalived akan assign floating ip ke master1 atau master2.
Lab Setup
- Pakai 3 node, masing-masing kuberi nama master0, master1, dan master2
- Network Subnet: 10.179.72.0/24
- Floating IP: 10.179.72.200/24
- master0 IP Address: 10.179.72.245/24
- master1 IP Address: 10.179.72.211/24
- master2 IP Address: 10.179.72.47/24
- Operating System: Debian 10 (Buster)
- KeepAlived version: v2.0.10 (11/12,2018)
- Haproxy version: 2.2.8-1~bpo10+1 2021/01/14
Note: Disini aku memakai keepalived dengan unicast, krn environment engga mendukung untuk multicast.
Install keepalived dan haproxy di semua node (master0, master1, dan master2)
Aku perlu menambahkan repo haproxy karena mau pakai versi 2.2 LTS yang belum masuk ke repo debian buster:
root@master0:~# echo deb http://deb.debian.org/debian buster-backports main > /etc/apt/sources.list.d/backports.list
Kemudian update list package:
root@master0:~# apt update
Lanjut install yang aku butuhkan:
root@master0:~# apt install curl keepalived haproxy=2.2.\* -t buster-backports
Kemudian aku perlu setup beberapa parameter kernel yang dibutuhkan oleh keepalived dan haproxy untuk bind floating ip/virtual ip.
root@master0:~# vi /etc/sysctl.d/999-keepalived-haproxy.conf
net.ipv4.ip_forward=1
net.ipv4.ip_nonlocal_bind=1
Aku apply parameter kernel tersebut tanpa reboot:
root@master0:~# sysctl -p
Ulangi semua langkah di node master1 dan master2
Konfigurasi keepalived dan haproxy di master0
Node Master0 akan aku jadikan master untuk keepalived, sedangkan master1 dan master2 akan aku jadikan backup untuk keepalived. Perlu di juga perhatikan bagian unicast_src_ip dan unicast_peer.
root@master0:~# vi /etc/keepalived/keepalived.conf
# konfigurasi yang aku pakai:
global_defs {
router_id LVS_DEVEL
}
vrrp_script check_apiserver {
script "/usr/bin/killall -0 haproxy"
interval 3
weight -2
fall 10
rise 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 101
unicast_src_ip 10.179.72.245
unicast_peer {
10.179.72.211
10.179.72.47
}
authentication {
auth_type PASS
auth_pass 42
}
virtual_ipaddress {
10.179.72.200
}
track_script {
check_apiserver
}
}
Konfigurasi keepalived dan haproxy di master1
Keepalived di master1 aku konfig sebagai backup dengan priority lebih rendah dari master0. Perlu diperhatikan bagian unicast_src_ip dan unicast_peer
root@master1:~# vi /etc/keepalived/keepalived.conf
# konfigurasi yang aku pakai:
global_defs {
router_id LVS_DEVEL
}
vrrp_script check_apiserver {
script "/usr/bin/killall -0 haproxy"
interval 3
weight -2
fall 10
rise 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
unicast_src_ip 10.179.72.211
unicast_peer {
10.179.72.245
10.179.72.47
}
authentication {
auth_type PASS
auth_pass 42
}
virtual_ipaddress {
10.179.72.200
}
track_script {
check_apiserver
}
}
Konfigurasi keepalived di master2
Keepalived di master1 aku konfig sebagai backup dengan priority lebih rendah dari master0 dan master1. Perlu diperhatikan bagian unicast_src_ip dan unicast_peer.
root@master2:~# vi /etc/keepalived/keepalived.conf
# konfigurasi yang aku pakai:
global_defs {
router_id LVS_DEVEL
}
vrrp_script check_apiserver {
script "/usr/bin/killall -0 haproxy"
interval 3
weight -2
fall 10
rise 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 99
unicast_src_ip 10.179.72.47
unicast_peer {
10.179.72.211
10.179.72.245
}
authentication {
auth_type PASS
auth_pass 42
}
virtual_ipaddress {
10.179.72.200
}
track_script {
check_apiserver
}
}
Perbedaan dari konfigurasi keepalived MASTER dan BACKUP:
[sumar@tingpret ~]$ diff master0-keepalived.conf master1-keepalived.conf
13c13
< state MASTER
---
> state BACKUP
16,17c16,17
< priority 101
< unicast_src_ip 10.179.72.245
---
> priority 100
> unicast_src_ip 10.179.72.211
20c20
< 10.179.72.211
---
> 10.179.72.245
[sumar@tingpret ~]$ diff master0-keepalived.conf master2-keepalived.conf
13c13
< state MASTER
---
> state BACKUP
16,17c16,17
< priority 101
< unicast_src_ip 10.179.72.245
---
> priority 99
> unicast_src_ip 10.179.72.47
21c21
< 10.179.72.47
---
> 10.179.72.245
[sumar@tingpret ~]$ diff master1-keepalived.conf master2-keepalived.conf
16,17c16,17
< priority 100
< unicast_src_ip 10.179.72.211
---
> priority 99
> unicast_src_ip 10.179.72.47
19a20
> 10.179.72.211
21d21
< 10.179.72.47
Yang membedakan adalah state , priority , unicast_src_ip , unicast_peer
Konfigurasi haproxy di semua node (master0, master1, dan master2)
Konfigurasi untuk haproxy adalah sama untuk semua node.
root@master0:~# vi /etc/haproxy/haproxy.cfg
# configurasi yang aku pakai:
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
# apiserver frontend, pakai ip floating
frontend apiserver
bind 10.179.72.200:8080
mode tcp
option tcplog
default_backend apiserver
# round robin balancing apiserver
backend apiserver
option httpchk GET /healthprobe
http-check expect status 200
mode tcp
balance roundrobin
server apiku1 10.179.72.2:80 check
server apiku2 10.179.72.3:80 check
server apiku3 10.179.72.4:80 check
Enable keepalived dan haproxy di semua node (master0, master1, dan master2)
root@master0:~# systemctl enable keepalived haproxy
root@master0:~# systemctl restart keepalived haproxy
root@master0:~# systemctl status keepalived haproxy
Ulangi untuk master1 dan master2.
Pengujian
Kondisi setelah semua jalan, terlihat bahwa floating ip (10.179.72.200) ada di master0:
+-------------+---------+----------------------+
| master0 | RUNNING | 10.179.72.245 (eth0) |
| | | 10.179.72.200 (eth0) |
+-------------+---------+----------------------+
| master1 | RUNNING | 10.179.72.211 (eth0) |
+-------------+---------+----------------------+
| master2 | RUNNING | 10.179.72.47 (eth0) |
+-------------+---------+----------------------+
Aku coba curl ke floating ip tersebut + port yang aku konfig di HAProxy:
[sumar@tingpret ~]$ curl -IL 10.179.72.200:8080
HTTP/1.1 200 OK
...
Vary: Accept-Encoding
Date: Wed, 31 Mar 2021 08:17:07 GMT
Good, request diteruskan ke apiserver di belakang haproxy.
Kemudian, coba matikan koneksi master0, atau shutdown nodenya juga bisa.
+-------------+---------+----------------------+
| master0 | STOPPED | |
+-------------+---------+----------------------+
| master1 | RUNNING | 10.179.72.211 (eth0) |
| | | 10.179.72.200 (eth0) |
+-------------+---------+----------------------+
| master2 | RUNNING | 10.179.72.47 (eth0) |
+-------------+---------+----------------------+
Terlihat floating ip di pindahkan ke node master1. Lalu aku coba lagi curl:
[sumar@tingpret ~]$ curl -IL 10.179.72.200:8080
HTTP/1.1 200 OK
...
Vary: Accept-Encoding
Date: Wed, 31 Mar 2021 08:20:17 GMT
Hasilnya good, request tetap diteruskan ke apiserver di belakang haproxy.
Sampai disini sudah selesai, semoga bermanfaat.
Cool~