<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>cloudrocket</title>
    <link>https://www.cloudrocket.at/</link>
    <description>Recent content on cloudrocket</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en-us</language>
    <lastBuildDate>Sat, 02 Jan 2021 21:27:01 +0100</lastBuildDate><atom:link href="https://www.cloudrocket.at/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Deploy an self hosted mail server with poste.io &amp; nginx proxy</title>
      <link>https://www.cloudrocket.at/posts/self-hosted-mail-server-with-poste.io-and-nginx/</link>
      <pubDate>Sat, 02 Jan 2021 21:27:01 +0100</pubDate>
      
      <guid>https://www.cloudrocket.at/posts/self-hosted-mail-server-with-poste.io-and-nginx/</guid>
      <description>&lt;h1 id=&#34;about&#34;&gt;about&lt;/h1&gt;
&lt;p&gt;&lt;a href=&#34;https://poste.io/&#34;&gt;poste.io&lt;/a&gt; is an fully featured self hosted email server which you can deploy in few minutes (nearly). There&amp;rsquo;s no magic rocket science in a black box, poste.io consists of bullet proof resilient &lt;a href=&#34;https://poste.io/doc/mailserver-parts&#34;&gt;parts&lt;/a&gt;&lt;/p&gt;
&lt;h1 id=&#34;basic-setup&#34;&gt;basic setup&lt;/h1&gt;
&lt;p&gt;To easily deploy the mail server follow the documentation on the &lt;a href=&#34;https://poste.io/doc/getting-started&#34;&gt;getting-started&lt;/a&gt; page or use an &lt;a href=&#34;https://gist.github.com/analogic/51fbe91b580d7913b72320f89bf994cc&#34;&gt;docker-compose&lt;/a&gt; template.&lt;/p&gt;
&lt;h2 id=&#34;before-you-start&#34;&gt;before you start&lt;/h2&gt;
&lt;p&gt;check that the following &lt;a href=&#34;https://poste.io/doc/configuring-dns&#34;&gt;DNS dependencies&lt;/a&gt; are met:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; A &amp;amp; CNAME records for mail server - e.g. &lt;code&gt;mail.yourdomain.com&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; MX record for mail server&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; &lt;a href=&#34;https://en.wikipedia.org/wiki/Sender_Policy_Framework&#34;&gt;TXT-SPF record for mail server&lt;/a&gt; - e.g. &lt;code&gt;v=spf1 mx ip4:YOUR_EXT_SERVER_IP -all&lt;/code&gt;.
You can test it with an &lt;a href=&#34;https://poste.io/spf&#34;&gt;SPF-Tester&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; &lt;a href=&#34;https://en.wikipedia.org/wiki/DMARC&#34;&gt;TXT-DMARC record for mail server&lt;/a&gt; - e.g. &lt;code&gt;v=DMARC1; p=none; rua=mailto:admin@YOUR_DOMAIN.COM; ruf=mailto:admin@YOUR_DOMAIN.COM&lt;/code&gt;.
You can test it with an &lt;a href=&#34;https://poste.io/dmarc&#34;&gt;DMARC-Tester&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;the-config&#34;&gt;the config&lt;/h2&gt;
&lt;p&gt;If you start with the &lt;a href=&#34;https://gist.github.com/analogic/51fbe91b580d7913b72320f89bf994cc&#34;&gt;gist-config&lt;/a&gt; maybe you stuck with the same problem like me.
I&amp;rsquo;m using an &lt;a href=&#34;https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion&#34;&gt;&amp;ldquo;frontend&amp;rdquo; nginx TLS proxy&lt;/a&gt; which handles the letsencrypt certificates in another docker container.
So i set up poste.io with external docker bridges &lt;a href=&#34;https://poste.io/doc/network-schemes&#34;&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2021/poste_network.png&#34; alt=&#34;Poste networks&#34; title=&#34;Poste.io docker networks&#34;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h2 id=&#34;the-tls-termination-problem&#34;&gt;the tls termination &amp;ldquo;problem&amp;rdquo;&lt;/h2&gt;
&lt;p&gt;At the first try poste.io started up &amp;amp; the proxy container requests the new letsencrypt certs &amp;amp; the https login to the mailserver worked out of the box.
The problem was that i forgot to link the certs from the proxy to the mail server. So the other TLS ports stucks with an default self signed cert.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2021/poste_ports.png&#34; alt=&#34;TLS ports&#34; title=&#34;Poste.io TLS without valid cert on first try&#34;&gt;&lt;/p&gt;
&lt;p&gt;To link the certs from the proxy to poste.io attach the following lines to your docker-compose.yml:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;...
    volumes:
      - NGINX_PROXY_PATH/ssl/certs/mail.YOUR_DOMAIN.com/key.pem:/data/ssl/server.key:ro
      - NGINX_PROXY_PATH/ssl/certs/mail.YOUR_DOMAIN.com/fullchain.pem:/data/ssl/ca.crt:ro
      - NGINX_PROXY_PATH/ssl/certs/mail.YOUR_DOMAIN.com/cert.pem:/data/ssl/server.crt:ro
...
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;the-final-config&#34;&gt;the final config&lt;/h2&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;version: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;3&amp;#39;&lt;/span&gt;

services:
  mailserver:
    image: analogic/poste.io:2
&lt;span style=&#34;color:#75715e&#34;&gt;#    container_name: mailserver&lt;/span&gt;
    restart: unless-stopped
    ports:
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;25:25&amp;#34;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;110:110&amp;#34;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;143:143&amp;#34;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;465:465&amp;#34;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;587:587&amp;#34;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;993:993&amp;#34;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;995:995&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#75715e&#34;&gt;#      - &amp;#34;4190:4190&amp;#34;&lt;/span&gt;
    environment:
      - HTTPS&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;OFF
      - LETSENCRYPT_EMAIL&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;admin@YOUR_DOMAIN.com
      - LETSENCRYPT_HOST&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;mail.YOUR_DOMAIN.com
      - VIRTUAL_HOST&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;mail.YOUR_DOMAIN.com
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./data:/data
      - NGINX_PROXY_PATH/ssl/html/.well-known:/opt/www/.well-known:ro
      - NGINX_PROXY_PATH/ssl/certs/mail.YOUR_DOMAIN.com/key.pem:/data/ssl/server.key:ro
      - NGINX_PROXY_PATH/ssl/certs/mail.YOUR_DOMAIN.com/fullchain.pem:/data/ssl/ca.crt:ro
      - NGINX_PROXY_PATH/ssl/certs/mail.YOUR_DOMAIN.com/cert.pem:/data/ssl/server.crt:ro
    networks:
      - frontend
      - default

networks:
  frontend:
    external:
      name: frontend_proxy
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;With this config the valid certs get linked from the nginx proxy to the mail server (which is pretty useful if you want to offer some TLS services).&lt;/p&gt;
&lt;h2 id=&#34;do-some-testing&#34;&gt;do some testing&lt;/h2&gt;
&lt;p&gt;If you now fire up the compose file, the TLS ports should be reached externaly with the valid TLS certs:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;color:#75715e&#34;&gt;# do some testing&lt;/span&gt;
HOST&lt;span style=&#34;color:#f92672&#34;&gt;=&lt;/span&gt;mail.YOUR_DOMAIN.com
&lt;span style=&#34;color:#75715e&#34;&gt;# test the SMTP port 587 with STARTTLS&lt;/span&gt;
echo | openssl s_client -servername $HOST -connect $HOST:587 -starttls smtp 2&amp;gt;/dev/null | openssl x509 -noout -issuer -subject -dates
&lt;span style=&#34;color:#75715e&#34;&gt;# test the SMTPS, IMAPS &amp;amp; POP3S ports&lt;/span&gt;
&lt;span style=&#34;color:#66d9ef&#34;&gt;for&lt;/span&gt; PORT in &lt;span style=&#34;color:#ae81ff&#34;&gt;465&lt;/span&gt; &lt;span style=&#34;color:#ae81ff&#34;&gt;993&lt;/span&gt; 995; &lt;span style=&#34;color:#66d9ef&#34;&gt;do&lt;/span&gt; echo | openssl s_client -servername $HOST -connect $HOST:$PORT 2&amp;gt;/dev/null | openssl x509 -noout -issuer -subject -dates; &lt;span style=&#34;color:#66d9ef&#34;&gt;done&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h1 id=&#34;security-recommendations&#34;&gt;security recommendations&lt;/h1&gt;
&lt;h2 id=&#34;optional-set-minimum-tls-12-for-clients&#34;&gt;(optional) set minimum TLS 1.2 for clients&lt;/h2&gt;
&lt;p&gt;If you use only &lt;em&gt;modern&lt;/em&gt; clients you safely can force TLS 1.2 as minimum for clients&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;docker exec -it CONTAINER_NAME bash
sed -i &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;s#inbound_min_version\ =\ &amp;#34;&amp;#34;#inbound_min_version\ =\ TLSv1.2#g&amp;#39;&lt;/span&gt; /data/server.ini
sed -i &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;s#inbound_ciphers\ =\ &amp;#34;&amp;#34;#inbound_ciphers\ =\ ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384#g&amp;#39;&lt;/span&gt; /data/server.ini
exit
docker restart CONTAINER_NAME
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;optional-configure-dkim-keys&#34;&gt;(optional) configure DKIM keys&lt;/h2&gt;
&lt;p&gt;DKIM keys will increase security &amp;amp; avoid spamming. To enable DKIM keys got to &amp;ldquo;Virtual domains → your-domain.com&amp;rdquo; &amp;amp; click on create keys.
This will auto generate your private keys and your DNS TXT public key entry which should look like this:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;_sYEARMMDD000._domainkey.your-domain.com IN TXT &amp;quot;k=rsa;p=......
&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2021/poste_dkim.png&#34; alt=&#34;DKIM&#34; title=&#34;configure DKIM keys&#34;&gt;&lt;/p&gt;
&lt;p&gt;Add an appropriate TXT entry to your DNS config &amp;amp; test it with an &lt;a href=&#34;https://poste.io/dkim?cloudrocket.at&amp;amp;s20210103707&#34;&gt;DKIM-Tester&lt;/a&gt;.
You also can trigger an full test with test emails =&amp;gt; &lt;a href=&#34;https://www.appmaildev.com/de/dkim&#34;&gt;https://www.appmaildev.com/de/dkim&lt;/a&gt;&lt;/p&gt;
&lt;h1 id=&#34;enjoy&#34;&gt;enjoy&lt;/h1&gt;
&lt;p&gt;&lt;img src=&#34;https://media.giphy.com/media/yidUzHnBk32Um9aMMw/giphy.gif&#34; alt=&#34;soo amazing fast&#34; title=&#34;soo amazing fast&#34;&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Monitor OpenWrt nodes with Prometheus</title>
      <link>https://www.cloudrocket.at/posts/monitor-openwrt-nodes-with-prometheus/</link>
      <pubDate>Mon, 28 Dec 2020 21:40:49 +0100</pubDate>
      
      <guid>https://www.cloudrocket.at/posts/monitor-openwrt-nodes-with-prometheus/</guid>
      <description>&lt;h1 id=&#34;motivation&#34;&gt;motivation&lt;/h1&gt;
&lt;p&gt;I&amp;rsquo;am using OpenWrt for a long time and monitoring of them was always a little bit tricky. It&amp;rsquo;s getting more complex if you&amp;rsquo;re going to use more access points. It&amp;rsquo;s an no brainer since the &lt;a href=&#34;https://openwrt.org/packages/table/start?dataflt%5BName_pkg-dependencies*~%5D=prometheus-node&#34;&gt;prometheus-node-exporter&lt;/a&gt; scripts exist.&lt;/p&gt;
&lt;h1 id=&#34;openwrt-setup&#34;&gt;openwrt setup&lt;/h1&gt;
&lt;h2 id=&#34;install-scripts&#34;&gt;install scripts&lt;/h2&gt;
&lt;p&gt;You should start with an up to date OpenWrt router. Install the prometheus node exporter scripts:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;prometheus-node-exporter-lua&lt;/li&gt;
&lt;li&gt;prometheus-node-exporter-lua-nat_traffic&lt;/li&gt;
&lt;li&gt;prometheus-node-exporter-lua-netstat&lt;/li&gt;
&lt;li&gt;prometheus-node-exporter-lua-openwrt&lt;/li&gt;
&lt;li&gt;prometheus-node-exporter-lua-wifi&lt;/li&gt;
&lt;li&gt;prometheus-node-exporter-lua-wifi_stations&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;a href=&#34;https://openwrt.org/packages/table/start?dataflt%5BName_pkg-dependencies*~%5D=prometheus-node&#34;&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2020/openwrt_prometheus_node_pkgs.png&#34; alt=&#34;packages&#34; title=&#34;OpenWrt Package list&#34;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h2 id=&#34;change-listening-interface&#34;&gt;change listening interface&lt;/h2&gt;
&lt;p&gt;If you use the default config, the node exporter can only be queried from the loopback interface.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;root@OpenWrt:/etc/config# cat prometheus-node-exporter-lua
config prometheus-node-exporter-lua &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;main&amp;#39;&lt;/span&gt;
	option listen_interface &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;loopback&amp;#39;&lt;/span&gt;
	option listen_ipv6 &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;0&amp;#39;&lt;/span&gt;
	option listen_port &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;9100&amp;#39;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For my setup i&amp;rsquo;ll change it to the lan network interface:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;root@OpenWrt:/etc/config# sed -i.bak &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;s#loopback#lan#g&amp;#39;&lt;/span&gt; /etc/config/prometheus-node-exporter-lua
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&amp;amp; restart the exporter&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;root@OpenWrt:/etc/config# /etc/init.d/prometheus-node-exporter-lua restart
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;optional-check&#34;&gt;(optional) check&lt;/h2&gt;
&lt;h4 id=&#34;--running-exporter-interface&#34;&gt;- running exporter interface&lt;/h4&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;root@OpenWrt:/etc/config# netstat -tulpn | grep &lt;span style=&#34;color:#ae81ff&#34;&gt;9100&lt;/span&gt;
tcp        &lt;span style=&#34;color:#ae81ff&#34;&gt;0&lt;/span&gt;      &lt;span style=&#34;color:#ae81ff&#34;&gt;0&lt;/span&gt; YOUR_ROUTER_IP:9100        0.0.0.0:*               LISTEN      3469/lua
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id=&#34;--running-exporter-config&#34;&gt;- running exporter config&lt;/h4&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;root@OpenWrt:/etc/config# ps | grep prometheus
 &lt;span style=&#34;color:#ae81ff&#34;&gt;3469&lt;/span&gt; root      &lt;span style=&#34;color:#ae81ff&#34;&gt;1920&lt;/span&gt; S    &lt;span style=&#34;color:#f92672&#34;&gt;{&lt;/span&gt;prometheus-node&lt;span style=&#34;color:#f92672&#34;&gt;}&lt;/span&gt; /usr/bin/lua /usr/bin/prometheus-node-exporter-lua --bind YOUR_ROUTER_IP --port &lt;span style=&#34;color:#ae81ff&#34;&gt;9100&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id=&#34;--open-port-from-host-in-lan&#34;&gt;- open port from host in lan&lt;/h4&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;user@HOST-IN-LAN ~&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;$ nmap YOUR_ROUTER_IP -p &lt;span style=&#34;color:#ae81ff&#34;&gt;9100&lt;/span&gt;
9100/tcp open  jetdirect
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id=&#34;--metrics-from-host-in-lan&#34;&gt;- metrics from host in lan&lt;/h4&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;user@HOST-IN-LAN ~&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;$ curl YOUR_ROUTER_IP:9100/metrics
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&amp;amp; you should see a lot of metrics here &amp;hellip;&lt;/p&gt;
&lt;h1 id=&#34;optional-prometheus-setup&#34;&gt;(optional) prometheus setup&lt;/h1&gt;
&lt;p&gt;You&amp;rsquo;re now ready to setup up Prometheus. If you already have an instance, you just need to add the &lt;a href=&#34;https://www.cloudrocket.at/posts/monitor-openwrt-nodes-with-prometheus/#prometheus-config&#34;&gt;scrape_config&lt;/a&gt; . If you haven&amp;rsquo;t one, no problem =&amp;gt;&lt;/p&gt;
&lt;h2 id=&#34;startup-new-instance&#34;&gt;startup new instance&lt;/h2&gt;
&lt;p&gt;If your&amp;rsquo;re familiar with docker &amp;amp; compose you can simply start up new prometheus/grafana instances. &lt;strong&gt;Use this config only for temporary usage&lt;/strong&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;version&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;2.1&amp;#39;&lt;/span&gt;

&lt;span style=&#34;color:#f92672&#34;&gt;networks&lt;/span&gt;:
  &lt;span style=&#34;color:#f92672&#34;&gt;monitor-net&lt;/span&gt;:
    &lt;span style=&#34;color:#f92672&#34;&gt;driver&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;bridge&lt;/span&gt;

&lt;span style=&#34;color:#f92672&#34;&gt;volumes&lt;/span&gt;:
    &lt;span style=&#34;color:#f92672&#34;&gt;prometheus_data&lt;/span&gt;: {}
    &lt;span style=&#34;color:#f92672&#34;&gt;grafana_data&lt;/span&gt;: {}

&lt;span style=&#34;color:#f92672&#34;&gt;services&lt;/span&gt;:

  &lt;span style=&#34;color:#f92672&#34;&gt;prometheus&lt;/span&gt;:
    &lt;span style=&#34;color:#f92672&#34;&gt;image&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;prom/prometheus:v2.23.0&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;container_name&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;prometheus&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;volumes&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;./prometheus:/etc/prometheus&lt;/span&gt;
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;prometheus_data:/prometheus&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;command&lt;/span&gt;:
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;--config.file=/etc/prometheus/prometheus.yml&amp;#39;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;--storage.tsdb.path=/prometheus&amp;#39;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;--web.console.libraries=/etc/prometheus/console_libraries&amp;#39;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;--web.console.templates=/etc/prometheus/consoles&amp;#39;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;--storage.tsdb.retention.time=200h&amp;#39;&lt;/span&gt;
      - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;--web.enable-lifecycle&amp;#39;&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;restart&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;unless-stopped&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;expose&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;9090&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;ports&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;9090&lt;/span&gt;:&lt;span style=&#34;color:#ae81ff&#34;&gt;9090&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;networks&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;monitor-net&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;labels&lt;/span&gt;:
      &lt;span style=&#34;color:#f92672&#34;&gt;org.label-schema.group&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;monitoring&amp;#34;&lt;/span&gt;

  &lt;span style=&#34;color:#f92672&#34;&gt;grafana&lt;/span&gt;:
    &lt;span style=&#34;color:#f92672&#34;&gt;image&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;grafana/grafana:7.3.6&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;container_name&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;grafana&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;volumes&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;grafana_data:/var/lib/grafana&lt;/span&gt;
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;./grafana/provisioning:/etc/grafana/provisioning&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;environment&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}&lt;/span&gt;
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}&lt;/span&gt;
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;GF_USERS_ALLOW_SIGN_UP=false&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;restart&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;unless-stopped&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;expose&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;3000&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;ports&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;3000&lt;/span&gt;:&lt;span style=&#34;color:#ae81ff&#34;&gt;3000&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;networks&lt;/span&gt;:
      - &lt;span style=&#34;color:#ae81ff&#34;&gt;monitor-net&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;labels&lt;/span&gt;:
      &lt;span style=&#34;color:#f92672&#34;&gt;org.label-schema.group&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;monitoring&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This is an trimmed down &amp;amp; modified version from &lt;a href=&#34;https://raw.githubusercontent.com/stefanprodan/dockprom/master/docker-compose.yml&#34;&gt;stefanprodan&lt;/a&gt;&lt;/p&gt;
&lt;h2 id=&#34;prometheus-config&#34;&gt;prometheus config&lt;/h2&gt;
&lt;p&gt;Before you spin up prometheus you have to change/add the config. Replace YOUR_ROUTER_IP:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;user@HOST-IN-LAN ~&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;$ vi prometheus/prometheus.yml 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;global&lt;/span&gt;:
  &lt;span style=&#34;color:#f92672&#34;&gt;scrape_interval&lt;/span&gt;:     &lt;span style=&#34;color:#ae81ff&#34;&gt;15s&lt;/span&gt;
  &lt;span style=&#34;color:#f92672&#34;&gt;evaluation_interval&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;15s&lt;/span&gt;

  &lt;span style=&#34;color:#75715e&#34;&gt;# Attach these labels to any time series or alerts when communicating with&lt;/span&gt;
  &lt;span style=&#34;color:#75715e&#34;&gt;# external systems (federation, remote storage, Alertmanager).&lt;/span&gt;
  &lt;span style=&#34;color:#f92672&#34;&gt;external_labels&lt;/span&gt;:
      &lt;span style=&#34;color:#f92672&#34;&gt;monitor&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;docker-host-alpha&amp;#39;&lt;/span&gt;

&lt;span style=&#34;color:#75715e&#34;&gt;# Load and evaluate rules in this file every &amp;#39;evaluation_interval&amp;#39; seconds.&lt;/span&gt;
&lt;span style=&#34;color:#f92672&#34;&gt;rule_files&lt;/span&gt;:
  - &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alert.rules&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#75715e&#34;&gt;# A scrape configuration containing exactly one endpoint to scrape.&lt;/span&gt;
&lt;span style=&#34;color:#f92672&#34;&gt;scrape_configs&lt;/span&gt;:
  - &lt;span style=&#34;color:#f92672&#34;&gt;job_name&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;openwrt-router1&amp;#39;&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;scrape_interval&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;5s&lt;/span&gt;
    &lt;span style=&#34;color:#f92672&#34;&gt;static_configs&lt;/span&gt;:
      - &lt;span style=&#34;color:#f92672&#34;&gt;targets&lt;/span&gt;: [&lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#39;YOUR_ROUTER_IP:9100&amp;#39;&lt;/span&gt;]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Your directory structure should be similar like this:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;user@HOST-IN-LAN dockprom&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;$ tree
.
├── docker-compose.yml
├── grafana
│   └── provisioning
└── prometheus
    └── prometheus.yml
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;start-containers&#34;&gt;start containers&lt;/h2&gt;
&lt;p&gt;Now start the docker containers in detached mode:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;user@HOST-IN-LAN dockprom&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;$ docker-compose up -d
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;After the images was pulled they should be started after a few seconds&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;color:#f92672&#34;&gt;[&lt;/span&gt;user@HOST-IN-LAN dockprom&lt;span style=&#34;color:#f92672&#34;&gt;]&lt;/span&gt;$ docker ps -a
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                       PORTS                    NAMES
ea8419f267e5        prom/prometheus:v2.23.0     &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;/bin/prometheus --c…&amp;#34;&lt;/span&gt;   &lt;span style=&#34;color:#ae81ff&#34;&gt;3&lt;/span&gt; hours ago         Up &lt;span style=&#34;color:#ae81ff&#34;&gt;3&lt;/span&gt; hours                   0.0.0.0:9090-&amp;gt;9090/tcp   prometheus
f17822ba6bcf        grafana/grafana:7.3.6       &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;/run.sh&amp;#34;&lt;/span&gt;                &lt;span style=&#34;color:#ae81ff&#34;&gt;3&lt;/span&gt; hours ago         Up &lt;span style=&#34;color:#ae81ff&#34;&gt;3&lt;/span&gt; hours                   0.0.0.0:3000-&amp;gt;3000/tcp   grafana
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Open the grafana dashboard in your browser =&amp;gt; &lt;a href=&#34;http://localhost:3000&#34;&gt;http://localhost:3000&lt;/a&gt;.
The default credentials are admin / admin. You&amp;rsquo;re prompted to change it.&lt;/p&gt;
&lt;h1 id=&#34;configure-grafana&#34;&gt;configure grafana&lt;/h1&gt;
&lt;h2 id=&#34;add-prometheus-datasource&#34;&gt;add prometheus datasource&lt;/h2&gt;
&lt;p&gt;Now add the new Prometheus instance as datasource to grafana. Go to &lt;a href=&#34;http://localhost:3000/datasources/new&#34;&gt;Configuration/Datasources&lt;/a&gt;, select the prometheus datasource &amp;amp; configure the URL. We can use the docker container name here because we are in the same docker network. If your run another setup, add here the ip of your main Prometheus server:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2020/openwrt_grafana_datasource.png&#34; alt=&#34;Datasource&#34; title=&#34;Prometheus datasource in Grafana&#34;&gt;&lt;/p&gt;
&lt;h2 id=&#34;add-openwrt-dashboard&#34;&gt;add OpenWrt dashboard&lt;/h2&gt;
&lt;p&gt;Now you can &lt;a href=&#34;http://localhost:3000/dashboard/import&#34;&gt;import the dashboard&lt;/a&gt;. You can use my &lt;a href=&#34;https://grafana.com/grafana/dashboards/11147&#34;&gt;Grafana dashboard for OpenWrt (11147)&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;a href=&#34;https://grafana.com/grafana/dashboards/11147&#34;&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2020/openwrt_grafana_dash1.png&#34; alt=&#34;Grafana dashboard&#34; title=&#34;Grafana dashboard&#34;&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Select the newly created Prometheus datasource.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2020/openwrt_grafana_dash2.png&#34; alt=&#34;Grafana dashboard&#34; title=&#34;Grafana dashboard&#34;&gt;&lt;/p&gt;
&lt;h1 id=&#34;watch-the-magic&#34;&gt;watch the magic&lt;/h1&gt;
&lt;p&gt;&lt;a href=&#34;http://localhost:3000/d/fLi0yXAWk/openwrt?orgId=1&amp;amp;refresh=30s&#34;&gt;&lt;img src=&#34;https://www.cloudrocket.at/images/2020/openwrt_magic.gif&#34; alt=&#34;Magic&#34; title=&#34;It&#39;s magic&#34;&gt;&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
  </channel>
</rss>
