By
Securitynik on 2020-10-05 12:58:01
Now that we have provided some basic security to Elasticsearch and Kibana communications, time to get some logs into the system.
Let's start this fourth post off with installing Metricbeat. Metricbeat can be downloaded directly from Elastic web site site. However, I prefer to use the instructions which can be found within my Kibana install.
From the "Add Data" page, I then selected "Elasticsearch metrics". As we are running on Ubuntu, the instruction below is from the "Deb" tab.
First download Metricbeat.
root@securitynik-monitoring:~$ cd /tmp/ root@securitynik-monitoring:~# curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.9.2-amd64.deb % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 37.2M 100 37.2M 0 0 23.4M 0 0:00:01 0:00:01 --:--:-- 23.4M root@securitynik-monitoring:/tmp$ ls metricbeat-7.9.2-amd64.deb metricbeat-7.9.2-amd64.deb
securitynik@securitynik-monitoring:/tmp$ sudo dpkg --install metricbeat-7.9.2-amd64.deb Selecting previously unselected package metricbeat. (Reading database ... 175727 files and directories currently installed.) Preparing to unpack metricbeat-7.9.2-amd64.deb ... Unpacking metricbeat (7.9.2) ... Setting up metricbeat (7.9.2) ... Processing triggers for systemd (245.4-4ubuntu3.2) ...
securitynik@securitynik-monitoring:/tmp$ cd /etc/metricbeat/ securitynik@securitynik-monitoring:/etc/metricbeat$ sudo cp metricbeat.yml metricbeat.yml.ORIGINAL
root@securitynik-monitoring:~# cat /etc/metricbeat/metricbeat.yml | grep --perl-regexp "^\s+host|^\s+protocol|^\s+username|^\s+password|^\s+protocol" host: "https://10.0.0.1:5601" hosts: ["https://10.0.0.1:9200"] protocol: "https" username: "elastic" password: "WelcomeToSecurityNikElastic"
root@securitynik-monitoring:~# tail --lines 8 /etc/metricbeat/metricbeat.yml # SSL Configuration enabled by Nik ssl.enabled: true output.elasticsearch.hosts: ["https://10.0.0.1:9200"] output.elasticsearch.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"] setup.kibana.ssl.enabled: true setup.kibana.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]
root@securitynik-monitoring:~# metricbeat modules enable elasticsearch Enabled elasticsearch Configuring the metricbeat Kibana module, I modified the "/etc/metricbeat/modules.d/elasticsearch.yml" to look as follow: root@securitynik-monitoring:~# vi /etc/metricbeat/modules.d/elasticsearch.yml root@securitynik-monitoring:~# cat /etc/metricbeat/modules.d/elasticsearch.yml # Module: elasticsearch # Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.9/metricbeat-module-elasticsearch.html - module: elasticsearch #metricsets: # - node # - node_stats period: 10s hosts: ["https://10.0.0.1:9200"] username: "elastic" password: "WelcomeToSecurityNikElastic" ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]
root@securitynik-monitoring:~# cat /etc/metricbeat/modules.d/kibana.yml # Module: kibana # Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.8/metricbeat-module-kibana.html - module: kibana #metricsets: # - status period: 10s hosts: ["https://10.0.0.1:5601"] #basepath: "" username: "elastic" password: "WelcomeToSecurityNikElastic"
root@securitynik-monitoring:/etc/metricbeat# metricbeat setup Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling. Index setup finished. Loading dashboards (Kibana must be running and reachable) Loaded dashboards
root@securitynik-monitoring:/etc/metricbeat# systemctl enable --now metricbeat.service Synchronizing state of metricbeat.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable metricbeat Created symlink /etc/systemd/system/multi-user.target.wants/metricbeat.service → /lib/systemd/system/metricbeat.service. root@securitynik-monitoring:/etc/metricbeat# systemctl status metricbeat.service ● metricbeat.service - Metricbeat is a lightweight shipper for metrics. Loaded: loaded (/lib/systemd/system/metricbeat.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2020-08-14 11:22:33 EDT; 8s ago Docs: https://www.elastic.co/products/beats/metricbeat Main PID: 33050 (metricbeat) Tasks: 11 (limit: 4563) Memory: 19.9M CGroup: /system.slice/metricbeat.service └─33050 /usr/share/metricbeat/bin/metricbeat -environment systemd -c /etc/metricbeat/metricbeat.yml -path.home /usr/share/metricbeat -path.config /etc/metric> Aug 14 11:22:38 securitynik-monitoring metricbeat[33050]: 2020-08-14T11:22:38.169-0400 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm p> Aug 14 11:22:38 securitynik-monitoring metricbeat[33050]: 2020-08-14T11:22:38.169-0400 INFO [index-management] idxmgmt/std.go:274 ILM policy success
root@securitynik-monitoring:/etc/metricbeat# metricbeat modules list Enabled: elasticsearch system Disabled: activemq aerospike apache ....
root@securitynik-monitoring:/etc/metricbeat# metricbeat modules enable beat kibana linux Enabled beat Enabled kibana Enabled linux
root@securitynik-monitoring:/etc/metricbeat# metricbeat setup --dashboards
Loading dashboards (Kibana must be running and reachable) Loaded dashboards