Monday, February 23, 2015

Dropbear SSH vulnerabilities in stock Arduino Yun. How to update to OpenSSH remotely.

The Arduino Yún ships with Dropbear version 2011.54-2. There are multiple vulnerabilities with it and is not advised to be used. If your arduino is in a remote location and you want to update to OpenSSH, without losing remote access to the device, follow these steps.

  • Change the Dropbear port to an unused/free one on your box and restart Dropbear
    uci set dropbear.@dropbear[0].Port=2222
    uci commit dropbear
    /etc/init.d/dropbear restart
  • Reconnect to your Yun via SSH on the configured port above
  • Install the openssh-server
    opkg update
    opkg install openssh-server
  • Enable and start OpenSSH server. OpenSSH will listen now on port 22
    /etc/init.d/sshd enable
    /etc/init.d/sshd start
  • Reconnect to your yun via SSH on port 22
  • Now you can disable Dropbear
    /etc/init.d/dropbear disable
    /etc/init.d/dropbear stop
  • Install the openssh-sftp-server package to install support for the SFTP protocol which SSHFS uses
    opkg update
    opkg install openssh-sftp-server
Log into the Yun from the web address. Go to Configure > Advanced Configuration > System > Software and under Installed Packages and remove Dropbear.

High (CVSS: 7.1)
NVT: Dropbear SSH Server Use-after-free Vulnerability (OID:
This host is installed with Dropbear SSH Server and is prone to a use-after-free vulnerability.
Vulnerability Detection Result
Vulnerability was detected according to the Vulnerability Detection Method.
This flaw allows remote authenticated users to execute arbitrary code and bypass command restrictions via multiple crafted command requests, related to channels concurrency.
Updates are available.
Affected Software/OS
Versions of Dropbear SSH Server 0.52 through 2011.54 are vulnerable.
Vulnerability Insight
A use-after-free vulnerability exists in Dropbear SSH Server 0.52 through 2011.54 when command restriction and public key authentication are enabled.
Vulnerability Detection Method
Check the version.
Details: Dropbear SSH Server Use-after-free Vulnerability (OID:
Version used: $Revision: 809 $
CVE: CVE-2012-0920
BID: 52159
Medium (CVSS: 5.0)
NVT: Dropbear SSH Server Multiple Security Vulnerabilities (OID:
This host is installed with Dropbear SSH Server and is prone to multiple vulnerabilities.
Vulnerability Detection Result
Vulnerability was detected according to the Vulnerability Detection Method.
The flaws allows remote attackers to cause a denial of service or to discover valid usernames.
Updates are available.
Affected Software/OS
Versions prior to Dropbear SSH Server 2013.59 are vulnerable.
Vulnerability Insight
Multiple flaws are due to, - The buf_decompress function in packet.c in Dropbear SSH Server before 2013.59 allows remote attackers to cause a denial of service (memory consumption) via a compressed packet that has a large size when it is decompressed. - Dropbear SSH Server before 2013.59 generates error messages for a failed logon attempt with different time delays depending on whether the user account exists.
Vulnerability Detection Method
Check the version.
Details: Dropbear SSH Server Multiple Security Vulnerabilities (OID:
Version used: $Revision: 809 $
CVE: CVE-2013-4421, CVE-2013-4434
BID: 62958, 62993
CERT: DFN-CERT-2013-1865 , DFN-CERT-2013-1772

Saturday, February 14, 2015

Graph Elertus and Google Nest in Nagios/check_mk

I recently got turned onto OMD/check_mk, which is probably the most user friendly tool for Nagios. It has a nearly complete Nagios stack, and more:

  • Nagios
  • Monitoring Plugins
  • check_nrpe
  • mrpe (a check_mk clone of nrpe)
  • Icinga
  • Shinken
  • NagVis
  • pnp4nagios
  • rrdtool/rrdcached
  • Check_MK
  • MK Livestatus
  • Multisite
  • Dokuwiki
  • Thruk
  • Mod-Gearman
  • check_logfiles
  • check_oracle_health
  • check_mysql_health
  • jmx4perl
  • check_webinject
  • check_multi

It has a very simple install and works with Debian/Ubuntu/Red Hat/SUSE. I recently deleted my Icinga build and went all in with this.

As well as regular servers, vmware and other things, I'm able to graph Google Nest's temperature, humidity and "leaf" status. You can set up alerts if it crosses a certain threshold and graph for as many years and you wish.

Check_MK with Google Nest

This is something that is sorely lacking in Google's current site since the graphing information is only stored for 10 days, and doesn't provide the granular level of detail I want.

Google Nest's graphs.

While I won't go into much detail on the install of OMD (it's pretty straightforward) and I won't go into much detail about the Nest integration (Great writeup on how to do that here)  I will be showing a way to get another product, the Elertus, monitored with check_mk as an example of what you can do with it.

The Elertus, is a wifi temperature, sound, light and water detector that runs on batteries. I wrote up an article about it on this blog before. It's a little trickier to monitor than the Nest due to the fact that there is no API, no way to scrape information off of the device since it only makes outbound connections.

Since my last post, last year, nothing has changed with it, nor has my opinion. It's still sluggish, no graphing, no "all clear" alerts... and a painfully slow (and useless) app to compliment. The plus side is that it's been almost a year and the batteries still work (AA batteries). I have had no connection issues from it as far as I can tell.

Other than an alerts tab, there's not much more to the Elertus website

When I had my Icinga server, I stood up another box that basically sniffed the traffic as it was in delivery to their servers.(which is still sent cleartext, btw). I cut out the bits and pieces I needed to get a basic graph up. Since I started this server, I decided to try a new method.

Using pfSense, I created a static DCHP connection for the device and an internal NAT rule to relay the traffic from the Elertus to my own web server running Apache with mod_dumpio turned on.

While this basically kills any communication with the Elertus servers, they won't be missed. With, my prior setup I was able to enjoy both my internally generated alerts, and their alerts, but I found my own to be a lot more useful.

The Elertus sends out a POST to their servers as a check in.

device_type=1&posix_time=1423925134&, ant:I, af:, pkt:l14_t298_h30_b70_m0_i1_e1_x-1_p1423925134, wdog:1, crtry:4, queue:3, ctime:w2285_d410_n130_s205_t3040, &

Apache has a virtualhost set up to receive those incoming PHP POST requests with dumpio enabled and the trace level set to 7.

<IfModule dumpio_module>
 DumpIOInput On
 DumpIOOutput On
 LogLevel dumpio:trace7

I also made sure to set custom logs, for just this module, as they will be filling up quick.

ErrorLog ${APACHE_LOG_DIR}/dumpio_module_error.log
CustomLog ${APACHE_LOG_DIR}/dumpio_module_access.log combined

The requests come in 2 at a time, sometimes more if there is any movement/light. So, to remedy the lack of a consistent time period to check the logs, I set up a cron job to pull the newest line in the log with POST data. This runs every 5 minutes.


tac /var/log/apache2/dumpio_module_error.log | grep -m1 "email" > /tmp/tempout.txt


The scripts within it, in turn, pull information out of the latest POST and updates the raw values to separate files. I've kept it modular so I can add more checks as I need them. I'm not too concerned about the water, movement and light alerts just yet.

The temperature is in Kelvin, so I converted it to Fahrenheit.


tempk=$(cat /tmp/tempout.txt | awk -F "=" '/light/ {print $8}' | sed 's/&.*//')
tempb=$(awk "BEGIN {print "$tempk" - 273.15}")
temp=$(echo ""$tempb"*1.8+32" | bc)

echo "$temp" > ~/perfdata/temp
(I know, this can be cleaned up a lot)

The battery and humidity, are pretty much the same thing, with different names. It just requires another {print $X} position. The light, movement and water sensor can be added just as easily since they are only values of 0 or 1.

Now, unlike Nagios, getting devices to check_mk is a breeze. Making custom checks, with RRD graphing is just as easy. You add the custom checks to the host itself, not the monitoring server.

This web server is Ubuntu and it has check_mk installed. There is a folder that allows you to put custom scripts to make local checks, above what it already monitors.


The setup is pretty straightforward if you want monitoring with performance graphs.


TEMP1=$(cat ~/perfdata/temp)

echo '<<<local>>>'
echo "P Temperature temp="$TEMP1";35:89;32:91;0;110"

Any script that is in this folder runs when check_mk runs and grabs the other server global readings.

The output appends itself to the bottom of the generated file, as a local check like this:

P Temperature temp=73.13;35:89;32:91;0;110

It tells check_mk that it's available to be added as a service, with graphing

  • P

The name of the service

  • Temperature

The variable name

  • temp=

The output from $TEMP1, the WARN min:max, the CRIT min:max and UNKNOWN lower;upper (for graphing reasons and not required). Notice that the colons and semicolons are there for a reason.

When you scan the host in WATO on the main check_mk site, the host should have basic performance graphs and automatically added to the notification ruleset the host was part of.

While it's not an ideal solution, it gets the job done. I wish Elertus would open their API up to developers, the would probably sell a lot more units if they did.

I recommend anyone who's given up with Nagios/Icinga to give OMD a try. It's pretty good. The documentation is a little poor (and mostly in German), but with Google Translate and some coffee, you can get through it if you've ever set up any other monitoring before.

Friday, February 13, 2015

How to enable check_mk for pfSense 2.2

The official check_mk plugin (v0.1.1) for pfSense 2.2 does not work. Here are the steps to configure it manually.

Edit the rc.conf (notice, this location is different from the official FreeBSD)

vi /etc/rc.conf.local

At the bottom of /etc/services, add the check_mk port definition.
vi /etc/services
 check_mk        6556/tcp   #check_mk agent

Add the tcp wrappers for check_mk
vi /etc/inetd.conf
 check_mk        stream  tcp     nowait          root    /usr/local/bin/check_mk_agent check_mk

Add the following and replace x.x.x.x with your ip
vi /etc/hosts.allow
 # Allow nagios server to access us
 check_mk_agent : x.x.x.x : allow
 check_mk_agent : ALL : deny

Adding custom inetd scripts to allow service control
vi /etc/rc.d/inetd
 # $FreeBSD$
 # PROVIDE: inetd
 # KEYWORD: shutdown
 . /etc/rc.subr
 load_rc_config $name
 run_rc_command "$1"

Make it executable
chmod +x /etc/rc.d/inetd

This is the only check_mk agent I could find that would work with pfSense 2.2. Replace all the text, or create a new file (if creating a new file, make sure to chmod +x it)
vi /usr/local/bin/check_mk_agent
# +------------------------------------------------------------------+
# |             ____ _               _        __  __ _  __           |
# |            / ___| |__   ___  ___| | __   |  \/  | |/ /           |
# |           | |   | '_ \ / _ \/ __| |/ /   | |\/| | ' /            |
# |           | |___| | | |  __/ (__|   <    | |  | | . \            |
# |            \____|_| |_|\___|\___|_|\_\___|_|  |_|_|\_\           |
# |                                                                  |
# | Copyright Mathias Kettner 2014    |
# +------------------------------------------------------------------+
# This file is part of Check_MK.
# The official homepage is at
# check_mk is free software;  you can redistribute it and/or modify it
# under the  terms of the  GNU General Public License  as published by
# the Free Software Foundation in version 2.  check_mk is  distributed
# in the hope that it will be useful, but WITHOUT ANY WARRANTY;  with-
# out even the implied warranty of  MERCHANTABILITY  or  FITNESS FOR A
# PARTICULAR PURPOSE. See the  GNU General Public License for more de-
# ails.  You should have  received  a copy of the  GNU  General Public
# License along with GNU Make; see the file  COPYING.  If  not,  write
# to the Free Software Foundation, Inc., 51 Franklin St,  Fifth Floor,
# Boston, MA 02110-1301 USA.

# Author: Lars Michelsen <>
#         Florian Heigl <>
#           (Added sections: df mount mem netctr ipmitool)

# NOTE: This agent has beed adapted from the Check_MK linux agent.
#       The most sections are commented out at the moment because
#       they have not been ported yet. We will try to adapt most
#       sections to print out the same output as the linux agent so
#       that the current checks can be used.

# This might be a good source as description of sysctl output:

# Remove locale settings to eliminate localized outputs where possible
export LC_ALL=C
unset LANG

export MK_LIBDIR="/usr/lib/check_mk_agent"
export MK_CONFDIR="/etc/check_mk"
export MK_TMPDIR="/var/run/check_mk"

# Make sure, locally installed binaries are found

# All executables in PLUGINSDIR will simply be executed and their
# ouput appended to the output of the agent. Plugins define their own
# sections and must output headers with '<<<' and '>>>'

# All executables in LOCALDIR will by executabled and their
# output inserted into the section <<<local>>>. Please refer
# to online documentation for details.

# close standard input (for security reasons) and stderr
#if [ "$1" = -d ]
#    set -xv
#    exec </dev/null 2>/dev/null

# Runs a command asynchronous by use of a cache file

echo '<<<check_mk>>>'
echo Version: 1.2.7i1
echo AgentOS: freebsd

osver="$(uname -r)"
is_jailed="$(sysctl -n security.jail.jailed)"

# Partitionen (-P verhindert Zeilenumbruch bei langen Mountpunkten)
# Achtung: NFS-Mounts werden grundsaetzlich ausgeblendet, um
# Haenger zu vermeiden. Diese sollten ohnehin besser auf dem
# Server, als auf dem Client ueberwacht werden.

echo '<<<df>>>'
# no special zfs handling so far, the ZFS.pools plugin has been tested to
# work on FreeBSD
if df -T > /dev/null ; then
    df -kTP -t ufs | egrep -v '(Filesystem|devfs|procfs|fdescfs|basejail)'
    df -kP -t ufs | egrep -v '(Filesystem|devfs|procfs|fdescfs|basejail)' | awk '{ print $1,"ufs",$2,$3,$4,$5,$6 }'

# Check NFS mounts by accessing them with stat -f (System
# call statfs()). If this lasts more then 2 seconds we
# consider it as hanging. We need waitmax.
#if type waitmax >/dev/null
#    STAT_VERSION=$(stat --version | head -1 | cut -d" " -f4)
#    STAT_BROKE="5.3.0"
#    echo '<<<nfsmounts>>>'
#    sed -n '/ nfs /s/[^ ]* \([^ ]*\) .*/\1/p' < /proc/mounts |
#        while read MP
#  do
#   if [ $STAT_VERSION != $STAT_BROKE ]; then
#      waitmax -s 9 2 stat -f -c "$MP ok %b %f %a %s" "$MP" || \
#    echo "$MP hanging 0 0 0 0"
#   else
#      waitmax -s 9 2 stat -f -c "$MP ok %b %f %a %s" "$MP" && \
#      printf '\n'|| echo "$MP hanging 0 0 0 0"
#   fi
#  done

# Check mount options.
# FreeBSD doesn't do remount-ro on errors, but the users might consider
# security related mount options more important.
echo '<<<mounts>>>'
mount -p -t ufs

# processes including username, without kernel processes
echo '<<<ps>>>'
if [ is_jailed = 0 ]; then
    ps ax -o state,user,vsz,rss,pcpu,command | sed -e 1d  -e '/\([^ ]*J\) */d' -e 's/*\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) */(\2,\3,\4,\5) /'
    ps ax -o user,vsz,rss,pcpu,command | sed -e 1d -e 's/ *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) */(\1,\2,\3,\4) /'

# Produce compatible load/cpu output to linux agent. Not so easy here.
echo '<<<cpu>>>'
echo `sysctl -n vm.loadavg | tr -d '{}'` `top -b -n 1 | grep -E '^[0-9]+ processes' | awk '{print $3"/"$1}'` `sysctl -n kern.lastpid` `sysctl -n hw.ncpu`

# Calculate the uptime in seconds since epoch compatible to /proc/uptime in linux
echo '<<<uptime>>>'
  up_seconds=$(( `date +%s` - `sysctl -n kern.boottime  | cut -f1 -d\, | awk '{print $4}'`))
idle_seconds=$(ps axw | grep idle | grep -v grep | awk '{print $4}' | cut -f1 -d\: )

# second value can be grabbed from "idle" process cpu time / num_cores
echo "$idle_seconds $up_seconds"

# Platten- und RAID-Status von LSI-Controlleren, falls vorhanden
#if which cfggen > /dev/null ; then
#   echo '<<<lsi>>>'
#   cfggen 0 DISPLAY | egrep '(Target ID|State|Volume ID|Status of volume)[[:space:]]*:' | sed -e 's/ *//g' -e 's/:/ /'

# Multipathing is supported in FreeBSD by now
if kldstat -v | grep g_multipath > /dev/null ; then
    echo '<<<freebsd_multipath>>>'
    gmultipath status | grep -v ^Name

# Soft-RAID
echo '<<<freebsd_geom_mirrors>>>'
gmirror status | grep -v ^Name

# Performancecounter Kernel
echo "<<<kernel>>>"
date +%s
forks=`sysctl -n vm.stats.vm.v_forks`
vforks=`sysctl -n vm.stats.vm.v_vforks`
rforks=`sysctl -n vm.stats.vm.v_rforks`
kthreads=`sysctl -n vm.stats.vm.v_kthreads`
echo "cpu" `sysctl -n kern.cp_time | awk ' { print $1" "$2" "$3" "$5" "$4 } '`
echo "ctxt" `sysctl -n vm.stats.sys.v_swtch`
echo "processes" `expr $forks + $vforks + $rforks + $kthreads`

# Network device statistics (Packets, Collisions, etc)
# only the "Link/Num" interface has all counters.
echo '<<<lnx_if:sep(58)>>>'
date +%s
if [ "$(echo $osver | cut -f1 -d\. )" -gt "8" ]; then
    netstat -inb | egrep -v '(^Name|plip|enc|pfsync|pflog|ovpns)' | grep Link | awk '{print"\t"$1":\t"$8"\t"$5"\t"$6"\t"$7"\t0\t0\t0\t0\t"$11"\t"$9"\t"$10"\t0\t0\t0\t0\t0"}'
    # pad output for freebsd 7 and before
    netstat -inb | egrep -v '(^Name|lo|plip)' | grep Link | awk '{print $1" "$7" "$5" "$6" 0 0 0 0 0 "$10" "$8" "$9" 0 0 "$11" 0 0"}'

# State of LSI MegaRAID controller via MegaCli.
# To install: pkg install megacli
if which MegaCli >/dev/null ; then
    echo '<<<megaraid_pdisks>>>'
    MegaCli -PDList -aALL -NoLog < /dev/null | egrep 'Enclosure|Raw Size|Slot Number|Device Id|Firmware state|Inquiry'
    echo '<<<megaraid_ldisks>>>'
    MegaCli -LDInfo -Lall -aALL -NoLog < /dev/null | egrep 'Size|State|Number|Adapter|Virtual'
    echo '<<<megaraid_bbu>>>'
    MegaCli -AdpBbuCmd -GetBbuStatus -aALL -NoLog < /dev/null | grep -v Exit

# OpenVPN Clients. 
# Correct log location unknown, sed call might also be broken
if [ -e /var/log/openvpn/openvpn-status.log ] ; then
    echo '<<<openvpn_clients:sep(44)>>>'
    sed -n -e '/CLIENT LIST/,/ROUTING TABLE/p' < /var/log/openvpn/openvpn-status.log  | sed -e 1,3d -e '$d' 

if which ntpq > /dev/null 2>&1 ; then
   echo '<<<ntp>>>'
   # remote heading, make first column space separated
   ntpq -np | sed -e 1,2d -e 's/^\(.\)/\1 /' -e 's/^ /%/'

# Checks for cups monitoring
#if which lpstat > /dev/null 2>&1; then
#  echo '<<<cups_queues>>>'
#  lpstat -p
#  echo '---'
#  for i in $(lpstat -p | grep -E "^(printer|Drucker)" | awk '{print $2}' | grep -v "@"); do
#    lpstat -o "$i"
#  done

# Heartbeat monitoring
#if which cl_status > /dev/null 2>&1; then
#  # Different handling for heartbeat clusters with and without CRM
#  # for the resource state
#  if [ -S /var/run/heartbeat/crm/cib_ro ]; then
#    echo '<<<heartbeat_crm>>>'
#    crm_mon -1 -r | grep -v ^$ | sed 's/^\s/_/g'
#  else
#    echo '<<<heartbeat_rscstatus>>>'
#    cl_status rscstatus
#  fi
#  echo '<<<heartbeat_nodes>>>'
#  for NODE in $(cl_status listnodes); do
#    if [ $NODE != $HOSTNAME ]; then
#      STATUS=$(cl_status nodestatus $NODE)
#      echo -n "$NODE $STATUS"
#      for LINK in $(cl_status listhblinks $NODE 2>/dev/null); do
#        echo -n " $LINK $(cl_status hblinkstatus $NODE $LINK)"
#      done
#      echo
#    fi
#  done

# Number of TCP connections in the various states
echo '<<<tcp_conn_stats>>>'
netstat -na | awk ' /^tcp/ { c[$6]++; } END { for (x in c) { print x, c[x]; } }'

# Postfix mailqueue monitoring
# Only handle mailq when postfix user is present. The mailq command is also
# available when postfix is not installed. But it produces different outputs
# which are not handled by the check at the moment. So try to filter out the
# systems not using postfix by searching for the postfix user.
# Cannot take the whole outout. This could produce several MB of agent output
# on blocking queues.
# Only handle the last 6 lines (includes the summary line at the bottom and
# the last message in the queue. The last message is not used at the moment
# but it could be used to get the timestamp of the last message.
#if which mailq >/dev/null 2>&1 && getent passwd postfix >/dev/null 2>&1; then
#  echo '<<<postfix_mailq>>>'
#  mailq | tail -n 6

#Check status of qmail mailqueue
#if type qmail-qstat >/dev/null
#   echo "<<<qmail_stats>>>"
#   qmail-qstat

# check zpool status
#if [ -x /sbin/zpool ]; then
#   echo "<<<zpool_status>>>"
#   /sbin/zpool status -x | grep -v "errors: No known data errors"

# Memory Usage
# currently we'll need sysutils/muse for this.
if [ -x /usr/local/bin/muse ]
echo '<<<mem>>>'
# yes, i don't know sed well.
muse -k 2>/dev/null | sed 's/Total/MemTotal/' | sed 's/Free/MemFree/'
swapinfo -k 1K | tail -n 1 | awk '{ print "SwapTotal: "$2" kB\nSwapFree: "$4" kB" }'

# Fileinfo-Check: put patterns for files into /etc/check_mk/fileinfo.cfg
if [ -r "$MK_CONFDIR/fileinfo.cfg" ] ; then
    echo '<<<fileinfo:sep(124)>>>'
    date +%s
    stat -f "%N|%z|%m" $(cat "$MK_CONFDIR/fileinfo.cfg")

Restart inetd
service inetd restart

You should be able to telnet into it it from the IP you specified in /etc/hosts.allow
telnet x.x.x.x 6556

If you cannot reach that port, allow the port in the firewall rules.

Go to Firewall > Rules

Click the LAN tab.

Create new rule.

Set the rule to pass traffic, on interface LAN, with the source IP of your check_mk server and to port 6556.

Give it a description, then save.

You should be able to pick up checks from OMD/check_mk.