DBA课程-day22-ELK第三天 DBA课程-day22-ELK第三天

昨天内容

索引优化 1.按照我们想要的效果生成索引 - 自定索引名称 - 按月生成 - 按域名和访问类型创建索引 - 能够索引日志里的每个字段内容2.filebeat配置 - nginx日志修改为json - filebeat添加模板相关选项 setup.template.name: "nginx" setup.template.pattern: "nginx_*" setup.template.enabled: false setup.template.overwrite: true - filebeat添加参数直接解析成json格式 json.keys_under_root: true json.overwrite_keys: true - filebeat的input添加tags标签 tags: ["www"] - filebeat的output里判断tags - index: "nginx_www_access-%{[beat.version]}-%{+yyyy.MM}" when.contains: tags: "www"3.收集tomcat日志 - 安装tomcat并访问，产生数据 - 修改tomcat日志为json格式，139行替换 - 验证tomcat日志是否为json了 - filebeat配置一模一样 -- 要使用*来匹配tomcat每天生成的日志 - type: log enabled: true paths: - /var/log/tomcat/localhost_access_log.*.txt json.keys_under_root: true json.overwrite_keys: true tags: ["tomcat"]4.收集java多行日志 - filebeat配置java日志路径 - 添加3行多行匹配的参数 - type: log enabled: true paths: - /var/log/elasticsearch/linux58.log tags: ["java"] multiline.pattern: '^\[' multiline.negate: true multiline.match: after5.kibana画图展示 - 柱状图，饼图，仪表图，折线图，Data Table,markdown - Dashboard大屏实时展示 - 查询时间要注意，查询条件对面板也是生效的 - 画完记得保存 - 别把es里的.kibana给删了

ELK介绍

ELk filebeat modules filebeat是go语言编写 - 快，不依赖于java环境

配置步骤： 1.配置相关modules参数

filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: true reload.period: 10s

2.查看模块

filebeat modules list

3.激活模块

[root@db01 ~]# filebeat modules enable nginx Enabled nginx

4.修改nginx为普通日志格式 5.修改filebeat配置文件

不足的地方： 1.错误日志和正确日志都混在一起了 2.不能按域名生成索引

【DBA课程-day22-ELK第三天】################################################################
filebeat modules 自定义索引和视图： 1.为了不影响实验，建议删除所有其他的索引

systemctl stop elasticsearch systemctl stop kibana rm -rf /data/elasticsearch/* rm -rf /var/lib/kibana/* systemctl start elasticsearch systemctl start kibana

2.修改nginx配置文件

sed -i 's#json#main#g' /etc/nginx/conf.d/bbs.conf

3.清空nginx日志

> /var/log/nginx/bbs_access.log

4.重启nginx

systemctl restart nginx

5.修改filebeat配置文件:

filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: true setup.kibana: host: "10.0.0.51:5601" output.elasticsearch: hosts: ["10.0.0.51:9200"] indices: - index: "nginx_bbs_access-%{[beat.version]}-%{+yyyy.MM}" when.contains: source: "/var/log/nginx/bbs_access.log" - index: "nginx_error-%{[beat.version]}-%{+yyyy.MM}" when.contains: fileset.name: "error" setup.template.name: "nginx" setup.template.pattern: "nginx_*" setup.template.enabled: false setup.template.overwrite: true

6.激活nginx模块报错

filebeat modules enable nginx

7.安装nginx modules插件

/usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/ingest-geoip-6.6.0.zip /usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/ingest-user-agent-6.6.0.zip

8.重启es

systemctl restart elasticsearch

9.修改模块配置

[root@db01 ~]# egrep -v "#|^$" /etc/filebeat/modules.d/nginx.yml - module: nginx access: enabled: true var.paths: ["/var/log/nginx/bbs_access.log"] error: enabled: true var.paths: ["/var/log/nginx/error.log"]

10.备份删除不必要的视图文件并导入到kibana

cp -a /usr/share/filebeat/kibana /root cd /usr/share/filebeat/kibana/6/dashboard find . -type f ! -name "*nginx*"|xargs rm -rf rm -rfml-nginx-* sed -i 's#filebeat\-\*#nginx\_\*#g' Filebeat-nginx-logs.json sed -i 's#filebeat\-\*#nginx\_\*#g' Filebeat-nginx-overview.json cd index-pattern/ sed -i 's#filebeat\-\*#nginx\_\*#g' filebeat.json filebeat setup --dashboards -E setup.dashboards.directory=/root/kibana/ rm -rf /var/lib/kibana/* systemctl restart kibana #########################################################################

安装docker步骤

rm -fr /etc/yum.repos.d/local.repo curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.ustc.edu.cn/docker-ce/linux/centos/docker-ce.repo sed -i 's#download.docker.com#mirrors.tuna.tsinghua.edu.cn/docker-ce#g' /etc/yum.repos.d/docker-ce.repo yum install docker-ce -y systemctl start docker cat > /etc/docker/daemon.json <



 
 下载镜像 
 docker pull nginx
docker run --name nginx -p 80:80 -d nginx
docker ps
docker logs -f nginx


 
 镜像：模板 
 容器：以什么模板启动的微型linuxdocker pull 镜像名:版本：拉去镜像
docker rmi镜像名或id：删除镜像
docker images：获取本地已有镜像
docker run镜像名或id：使用默认端口启动镜像
docker run -p port:port1 -d 镜像名或id：指定port端口映射到port1，并后台启动镜像进入容器内部 docker exec -it nginx容器ID /bin/bash
docker ps：查看正在运行的docker容器
docker ps -a：查看所有执行过run命令的容器服务（包括已经停止的容器）
docker stop 容器id：停止某个容器
docker restart 容器id：重启某个容器
docker rm 容器Id：删除某个容器docker exec -it nginx /bin/bash[root@db03 ~]# docker images
REPOSITORYTAGIMAGE IDCREATEDSIZE
nginxlatestf68d6e55e06510 days ago109MB[root@db03 ~]# docker ps
CONTAINER IDIMAGECOMMANDCREATEDSTATUSPORTSNAMES
0fe1c72d0bcdnginx"nginx -g 'daemon of…"3 hours agoUp 3 hours0.0.0.0:80->80/tcpnginx[root@db03 ~]# docker ps -a
CONTAINER IDIMAGECOMMANDCREATEDSTATUSPORTSNAMES
0fe1c72d0bcdnginx"nginx -g 'daemon of…"3 hours agoUp 3 hours0.0.0.0:80->80/tcpnginx


 
 收集docker容器日志 
 1.生成多个容器 
 systemctl stop nginx 
docker stop $(docker ps -q)
docker rm $(docker ps -aq)
docker run --name nginx -p 80:80 -d nginx
docker commit nginx nginx:v2
docker run --name mysql -p 8080:80 -d nginx:v2
docker images
docker ps 
docker logs -f nginx
docker logs -f mysql


 
 2.修改filebeat配置文件 
 filebeat.inputs:
- type: docker
containers.ids: 
- '*'
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false 
setup.kibana:
host: "10.0.0.51:5601"
output.elasticsearch:
hosts: ["10.0.0.51:9200"]
index: "docker-nginx-%{[beat.version]}-%{+yyyy.MM}"
setup.template.name: "docker"
setup.template.pattern: "docker-*"
setup.template.enabled: false
setup.template.overwrite: true


 
 3.重启filebeat 
 systemctl restart filebeat原始的docker容器日志
{
"log": "2019/07/12 07:58:46 [error] 6#6: *3 open() \"/usr/share/nginx/html/lalalala\" failed (2: No such file or directory), client: 10.0.0.1, server: localhost, request: \"GET /lalalala HTTP/1.1\", host: \"10.0.0.53\"\n",
"stream": "stderr",
"time": "2019-07-12T07:58:46.805441064Z"
}增加lable之后的日志
{
"log": "10.0.0.1 - - [12/Jul/2019:08:52:59 +0000] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Windows NT 6.1;
 Win64;
 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36\" \"-\"\n",
"stream": "stdout",
"attrs": {
"service": "mysql"
},
"time": "2019-07-12T08:52:59.944284567Z"
}


 
 配置步骤： 
 按服务类型拆分docker容器日志 
 1.安装docker-compose 
 yum install -y python2-pip


 
 2.这里使用pip安装，默认源为国外，可以使用国内加速，相关网站 
 https://mirrors.tuna.tsinghua.edu.cn/help/pypi/
pip加速操作命令
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple pip -U
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple


 
 3.继续安装docker-compose 
 pip install docker-compose


 
 4.检查版本 
 docker-compose version


 
 5.创建docker-compose配置文件 
 [root@db03 ~]# cat docker-compose.yml 
version: '3'
services:
nginx:
image: nginx:latest
#设置labels
labels:
service: nginx
#logging设置增加labels.service
logging:
options:
labels: "service"
ports:
- "80:80"
mysql:
image: nginx:v2
#设置labels
labels:
service: mysql
#logging设置增加labels.service
logging:
options:
labels: "service"
ports:
- "8080:80"


 
 6.使用docker-compose启动docker 
 docker stop $(docker ps -q)
docker rm $(docker ps -aq)
docker-compose up -d
docker ps 


 
 7.修改filebeat配置文件 
 filebeat.inputs:
- type: log 
enabled: true 
paths:
- /var/lib/docker/containers/*/*-json.log
json.keys_under_root: true
json.overwrite_keys: true
setup.kibana:
host: "10.0.0.51:5601"
output.elasticsearch:
hosts: ["10.0.0.51:9200"]
indices:
- index: "docker-nginx-access-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "nginx"
stream: "stdout"
- index: "docker-nginx-error-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "nginx"
stream: "stderr"
- index: "docker-mysql-access-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "mysql"
stream: "stdout"
- index: "docker-mysql-error-%{[beat.version]}-%{+yyyy.MM}"
when.contains:
attrs.service: "mysql"
stream: "stderr"setup.template.name: "docker"
setup.template.pattern: "docker-*"
setup.template.enabled: false
setup.template.overwrite: true


 
 filebeat配置 
 filebeat.inputs:
- type: log
enabled: true 
paths:
- /var/log/nginx/bbs_access.log
json.keys_under_root: true
json.overwrite_keys: true
tags: ["bbs"]
- type: log
enabled: true 
paths:
- /var/log/nginx/www_access.log
json.keys_under_root: true
json.overwrite_keys: true
tags: ["www"]
setup.kibana:
host: "10.0.0.51:5601"
output.redis:
hosts: ["localhost"]
keys:
- key: "bbs"
when.contains:
tags: "bbs"
- key: "www"
when.contains:
tags: "www"
db: 0
timeout: 5setup.template.name: "nginx"
setup.template.pattern: "nginx-*"
setup.template.enabled: false
setup.template.overwrite: true


 
 redis查看命令 
 redis-cli 
keys *
llen bbs
llen www


 
 logstash配置 
 [root@db01 /data/soft]# cat /etc/logstash/conf.d/redis.conf 
input {
redis {
host => "127.0.0.1"
port => "6379"
db => "0"
key => "bbs"
data_type => "list"
}redis {
host => "127.0.0.1"
port => "6379"
db => "0"
key => "www"
data_type => "list"
}
}
#filter {
#mutate {
#convert => ["upstream_time", "float"]
#convert => ["request_time", "float"]
#}
#}output {
if "bbs" in [tags] {
stdout {} 
elasticsearch {
hosts => "http://10.0.0.51:9200"
manage_template => false
index => "nginx-bbs-%{+yyyy.MM}"
}
}if "www" in [tags] {
stdout {} 
elasticsearch {
hosts => "http://10.0.0.51:9200"
manage_template => false
index => "nginx-www-%{+yyyy.MM}"
}
}
}


 
 logstash启动命令 
 /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/redis.conf


 
 报错记录1 
 [root@db03 ~]# docker run --name nginx -p 80:80 -d nginx
docker: Error response from daemon: Conflict. The container name "/nginx" is already in use by container "0fe1c72d0bcdc9296d540f6b5aded9861407d81732b3063f615e279d293c5573". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.


 
 报错记录2 
 Error response from daemon: driver failed programming external connectivity on endpoint nginx (dc4f838af1cd2f79ee9b422ea0bb2a2c3f1831c86ba6ed60d298ca266b2cc7ef): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use
Error: failed to start containers: nginx


 
 报错记录3 
 [root@db03 ~]# docker rm $(docker ps -q)
Error response from daemon: You cannot remove a running container 9bab465e57c3fe165367595472b61bdd64e4b84010ef5a7f54757b5946a23d09. Stop the container before attempting removal or force remove
Error response from daemon: You cannot remove a running container 0fe1c72d0bcdc9296d540f6b5aded9861407d81732b3063f615e279d293c5573. Stop the container before attempting removal or force remove


 
 filebeat modules报错1 
 现象：
执行filebeat modules list提示参数没有配置
[root@db01 ~]# filebeat modules list
Error in modules manager: modules management requires 'filebeat.config.modules.path' setting原因：
配置文件里没有配置相关参数，导致找不到modules路径解决：
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 10s


 
 filebeat modules报错2： 
 现象：激活nginx模块报错
2019-07-12T09:22:06.599+0800ERRORfileset/factory.go:142Error loading pipeline: Error loading pipeline for fileset nginx/access: This module requires the following Elasticsearch plugins: ingest-user-agent, ingest-geoip. You can install them by running the following commands on all the Elasticsearch nodes:
sudo bin/elasticsearch-plugin install ingest-user-agent
sudo bin/elasticsearch-plugin install ingest-geoip#注意！！！放在root下！！！
/usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/ingest-geoip-6.6.0.zip 
/usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/ingest-user-agent-6.6.0.zip 


 
 filebeat modules报错3： 
 filebeat setup -e
2019-07-12T09:41:32.833+0800ERRORinstance/beat.go:911Exiting: fail to create the Kibana loader: Error creating Kibana client: Error creating Kibana client: fail to get the Kibana version: HTTP GET request to /api/status fails: fail to execute the HTTP GET request: Get http://localhost:5601/api/status: dial tcp 127.0.0.1:5601: connect: connection refused. Response: .
Exiting: fail to create the Kibana loader: Error creating Kibana client: Error creating Kibana client: fail to get the Kibana version: HTTP GET request to /api/status fails: fail to execute the HTTP GET request: Get http://localhost:5601/api/status: dial tcp 127.0.0.1:5601: connect: connection refused. Response: .原因：
filebeat配置文件里没有配置kibana相关参数，导致使用默认的localhost:5601解决：filebeat添加相关配置
setup.kibana:
host: "10.0.0.51:5601"


 
 安装docker-compose报错1 
 ERROR: Cannot uninstall 'requests'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.解决方法：
pip install --ignore-installed requests




		  	

    
    




    
    
    


推荐阅读

           
                  
              
                  车子低速起步咔咔响 车辆低速行驶有咔咔响 
                
                   
                
              
            

                  
              
                  快速提高情商的5大方法 
                
                   
                
              
            

                  
              
                  mongodb参数 mongodb 函数性能 
                
                   
                
              
            

                  
              
                  为什么斗鱼的服务器消失了？ 斗鱼怎么看不见服务器了 
                
                   
                
              
            

                  
              
                  超好听的男生女生网名 独特好听男生女生网名 
                
                   
                
              
            

                  
              
                  龟苓膏和烧仙草的区别有什么 
                
                   
                
              
            

                  
              
                  肠粉添加硼砂的比例 请查看答案 
                
                   
                
              
            

                  
              
                  jquery自适应位置，js自适应 
                
                   
                
              
            

                  
              
                  联想电视屏幕出现竖线维修报价 
                
                   
                
              
            

                  
              
                  香椿用热水烫完之后用过凉水吗 
                
                   
                
              
            

                  
              
                  男性包皮过长是什么原因造成的 
                
                   
                
              
            

                  
              
                  iphone14pro暗夜紫色，Iphone屏幕上方出现暗紫色圆圈 
                
                   
                
              
            

                  
              
                  三星tab s6 lite 三星tab s 
                
                   
                
              
            

                  
              
                  进天国的七大条件 天国在哪里 
                
                   
                
              
            

                  
              
                  加湿器里面可以加香薰精油吗 
                
                   
                
              
            

                  
              
                  极左是什么意思 极右是什么意思 
                
                   
                
              
            

                  
              
                  牛皮 
                
                   
                
              
            

                  
              
                  猪肉价格今日价最新价 
                
                   
                
              
            

                  
              
                  新车贴车衣的好处和坏处有哪些 新车贴车衣的好处和坏处 
                
                   
                
              
            

                  
              
                  什么叫危机公关 
                
                   
                
              
            

          

课后分享记录 
 机器学习|机器学习 Andrew Ng《Machine Learning》课程笔记1 
 20180604 
 学习拆书的5天收获。 
 易效能进阶课程笔记29 
 思维导图课程总结 
 20171110-过敏性疾病的防治 
 万建峰老师课程感受 
 Paul|Paul 《商业教练》课程学习回顾（1） 
 分享校本课程开发历程，点燃校本课程开发激情