因为某种原因必须要开启SElinux，用yum装了一个zabbix-agent 替换了一下配置文件结果悲剧了
# systemctl restart zabbix-agent
● zabbix-agent.service - Zabbix Agent
Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Thu 2019-04-11 10:28:32 UTC; 10s ago
Process: 5234 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=0/SUCCESS)
Process: 5216 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 5218 (code=exited, status=0/SUCCESS)
Apr 11 10:28:31 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: Starting Zabbix Agent...
Apr 11 10:28:31 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: zabbix-agent.service: Supervising process 5218 which is not our child. We'll most likely not notice when it exits.
Apr 11 10:28:31 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: Started Zabbix Agent.
Apr 11 10:28:32 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: Stopping Zabbix Agent...
Apr 11 10:28:32 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: start request repeated too quickly for zabbix-agent.service
Apr 11 10:28:32 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: Failed to start Zabbix Agent.
Apr 11 10:28:32 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: Unit zabbix-agent.service entered failed state.
Apr 11 10:28:32 ip-172-16-210-219.us-west-2.compute.internal systemd[1]: zabbix-agent.service failed.
第一时间想到是不是selinux的问题
# getenforce
Enforcing
# setenforce 0
设置成selinux Permissive模式发现就好了
# tail -f /var/log/messages |grep zabbix
Apr 11 10:44:23 ip-172-16-210-243 zabbix_agentd: zabbix_agentd [16785]: cannot open config file "/etc/zabbix/zabbix_agentd.conf": [13] Permission denied
Apr 11 10:44:23 ip-172-16-210-243 systemd: zabbix-agent.service: control process exited, code=exited status=1
Apr 11 10:44:23 ip-172-16-210-243 systemd: Unit zabbix-agent.service entered failed state.
Apr 11 10:44:23 ip-172-16-210-243 systemd: zabbix-agent.service failed.
可以看到是/etc/zabbix/zabbix_agentd.conf这个文件的问题
# tail -f /var/log/audit/audit.log |grep zabbix
type=AVC msg=audit(1554979672.948:2028728): avc: denied { read } for pid=19011 comm="zabbix_agentd" name="zabbix_agentd.conf" dev="nvme0n1p1" ino=4196160 scontext=system_u:system_r:zabbix_agent_t:s0 tcontext=unconfined_u:object_r:admin_home_t:s0 tclass=file permissive=0
type=SYSCALL msg=audit(1554979672.948:2028728): arch=c000003e syscall=2 success=no exit=-13 a0=55e256f55190 a1=0 a2=1b6 a3=24 items=0 ppid=1 pid=19011 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="zabbix_agentd" exe="/usr/sbin/zabbix_agentd" subj=system_u:system_r:zabbix_agent_t:s0 key=(null)
type=SERVICE_START msg=audit(1554979672.955:2028729): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=zabbix-agent comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
基本上可以定位问题了zabbix_agentd.conf 跟zabbix_agentd进程的安全上下文不符合所以没权限读取zabbix_agentd.conf文件
先看一下文件和进程的安全上下文类型
# ls -Z /etc/zabbix/zabbix_agentd.conf
-rw-r--r--. root root unconfined_u:object_r:admin_home_t:s0 /etc/zabbix/zabbix_agentd.conf
# semanage fcontext -l |grep zabbix
/var/log/zabbix.* all files system_u:object_r:zabbix_log_t:s0
/etc/zabbix/web(/.*)? all files system_u:object_r:httpd_sys_rw_content_t:s0
/var/lib/zabbix(/.*)? all files system_u:object_r:zabbix_var_lib_t:s0
/var/run/zabbix(/.*)? all files system_u:object_r:zabbix_var_run_t:s0
/etc/rc.d/init.d/(zabbix|zabbix-server) regular file system_u:object_r:zabbix_initrc_exec_t:s0
/var/lib/zabbixsrv(/.*)? all files system_u:object_r:zabbix_var_lib_t:s0
/usr/lib/zabbix/externalscripts(/.*)? all files system_u:object_r:zabbix_script_exec_t:s0
/var/lib/zabbix/externalscripts(/.*)? all files system_u:object_r:zabbix_script_exec_t:s0
/usr/bin/zabbix_server regular file system_u:object_r:zabbix_exec_t:s0
/usr/bin/zabbix_agentd regular file system_u:object_r:zabbix_agent_exec_t:s0
/usr/sbin/zabbix_proxy regular file system_u:object_r:zabbix_exec_t:s0
/usr/sbin/zabbix_agentd regular file system_u:object_r:zabbix_agent_exec_t:s0
/usr/sbin/zabbix_server regular file system_u:object_r:zabbix_exec_t:s0
/usr/sbin/zabbix_proxy_mysql regular file system_u:object_r:zabbix_exec_t:s0
/usr/sbin/zabbix_proxy_pgsql regular file system_u:object_r:zabbix_exec_t:s0
/usr/sbin/zabbix_server_mysql regular file system_u:object_r:zabbix_exec_t:s0
/usr/sbin/zabbix_server_pgsql regular file system_u:object_r:zabbix_exec_t:s0
/etc/rc.d/init.d/zabbix-agentd regular file system_u:object_r:zabbix_agent_initrc_exec_t:s0
/usr/sbin/zabbix_proxy_sqlite3 regular file system_u:object_r:zabbix_exec_t:s0
/usr/sbin/zabbix_server_sqlite3 regular file system_u:object_r:zabbix_exec_t:s0
解决方法
主要是文件的安全上下文类型不符合，用systemd启动的服务会有问题 直接 zabbix-agent -c /etc/zabbix/zabbix_agentd.conf 启动是没问题的
# chcon -t etc_t /etc/zabbix/zabbix_agentd.conf
# chcon -u system_u /etc/zabbix/zabbix_agentd.conf # 非必要
# ls -Z /etc/zabbix/zabbix_agentd.conf
-rw-r--r--. root root system_u:object_r:etc_t:s0 /etc/zabbix/zabbix_agentd.conf
或者直接从别的文件拷贝安全上下文设置
# chcon --reference=/etc/zabbix/zabbix_agentd.d/userparameter_mysql.conf /etc/zabbix/zabbix_agentd.conf
问题二在/etc/zabbix/zabbix_agentd.conf加了一个配置
UserParameter=lvm.used,sudo -u root lvs |awk '{sum=$5+$6} END {print sum}'
测试一下
# zabbix_get -s 172.16.210.219 -k lvm.used
sh: /usr/bin/sudo: Permission denied
已经确定有zabbix 用户有sudo权限，那应该还是selinux的问题
# tail -f /var/log/audit/audit.log |grep zabbix
type=SYSCALL msg=audit(1554981718.821:2030328): arch=c000003e syscall=59 success=no exit=-13 a0=24abe30 a1=24ac490 a2=24aada0 a3=7ffeca2d9e20 items=0 ppid=40804 pid=40805 auid=4294967295 uid=993 gid=989 euid=993 suid=993 fsuid=993 egid=989 sgid=989 fsgid=989 tty=(none) ses=4294967295 comm="sh" exe="/usr/bin/bash" subj=system_u:system_r:zabbix_agent_t:s0 key=(null)
type=AVC msg=audit(1554981718.821:2030329): avc: denied { execute } for pid=40805 comm="sh" name="sudo" dev="nvme0n1p1" ino=13122559 scontext=system_u:system_r:zabbix_agent_t:s0 tcontext=system_u:object_r:sudo_exec_t:s0 tclass=file permissive=0
type=SYSCALL msg=audit(1554981718.821:2030329): arch=c000003e syscall=21 success=no exit=-13 a0=24abe30 a1=1 a2=7ffeca2da8a0 a3=7ffeca2d9e20 items=0 ppid=40804 pid=40805 auid=4294967295 uid=993 gid=989 euid=993 suid=993 fsuid=993 egid=989 sgid=989 fsgid=989 tty=(none) ses=4294967295 comm="sh" exe="/usr/bin/bash" subj=system_u:system_r:zabbix_agent_t:s0 key=(null)
因为涉及到系统组件这里就不推荐修改文件的安全上下文了
# semanage permissive -a zabbix_agent_t #把zabbix进程加入宽松模式
# semanage permissive -d zabbix_agent_t #删除