Proxmox with an Intermediate cert

Proxmox testing node running with an Intermediate Certificate Authority cert

It has been a long time since my last post. My boxes have been working fine so far and up until yesterday I had not noticed any issues. After updating JAVA on my machine I started to get errors concerning invalid certificates. I had previously installed new proper certs on my box so that might have been the cause.

Regarding my certs: I use the built in tools on pfSense to generate and manage all certs that I use on testing units. There I have a Root Certificate Authority setup and its cert is installed on the machines I use to debug my test installations (to avoid paying for temporary and easily disposable certs). On it I had created a server cert for my Proxmox testing node and had it installed a long time ago.

Looking for solutions I came across a post on how installing a intermediate certificate authority on the proxmox node could solve this and here is how I did it:

First I backed up all my old certs:

mv /etc/pve/pve-root-ca.pem /etc/pve/pve-root-ca.pem.bak 
mv /etc/pve/pve-www.key /etc/pve/pve-www.key.bak 
mv /etc/pve/priv/pve-root-ca.key /etc/pve/priv/pve-root-ca.key.bak 
mv /etc/pve/priv/pve-root-ca.srl /etc/pve/priv/pve-root-ca.srl.bak 
mv /etc/pve/local/pve-ssl.key /etc/pve/local/pve-ssl.key.bak 
mv /etc/pve/local/pve-ssl.pem /etc/pve/local/pve-ssl.pem.bak

Then I regenerated them and restarted all pvedaemon and pveproxy services:

pvecm updatecerts --force
service pvedaemon restart 
service pveproxy restart

I proceeded creating a new Intermediate Certificate Authority and a Server Certificate on my pfSense going on System > Cert Manager > CA > Add. Filled in the details and then Cert Manager > Certificate > Add and selected the previously intermediate cert authority. Downloaded the server key and cert and the authority cert.

Here came the tricky part:

  • The certificate authority cert became /etc/pve/pve-root-ca.pem;
  • The server key was copied to /etc/pve/local/pve-ssl.key; and
  • The server cert was edited to include the certificate authority cert at the bottom and copied to /etc/pve/local/pve-ssl.pem.

Restarted the services again and tested:

service pvedaemon restart 
service pveproxy restart

All working fine now!

Thanks to symmcom on the Proxmox forums and the maintainers of the Proxmox Wiki for some of these tips!

Cheers!

Updating Puppet on Debian 6

Some missing dependencies stopped Puppet from automatically updating on my systems.

After some research, I found the proper way to enable Puppet Labs Repos on my installs:

wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb && sudo dpkg -i puppetlabs-release-precise.deb

proceeded by

sudo apt-get update && sudo apt-get dist-upgrade -y && sudo apt-get upgrade -y

Will cause everything to be running on the latest version.

Automatically shutdown FreeNAS box when all clients are offline

I’ve been struggling with trying to keep the power consumption of my FreeNAS box to a minimum for quite a while now.

On the past months I also started to play with the Raspberry Pi, running it as a media center device primarely using OpenElec at first and then Raspbmc later on.

I intend to talk about those experiences on a separate posting. At first what concerned me is that for my media center to work I would need to keep my file server running all the time, something that I would not like to do.

My solution comes in two parts:

First to build a device to monitor my lan, pinging known addresses (statically issued by my local DHCP server) and sending a Wake-On-Lan packet when one of them come online.

Second, to keep monitoring my lan, checking every minute if such devices are still present, and if they are not, shut down my file server after 30 minutes.

Now imagine the following scenario: I arrive at home with my phone and laptop. My phone upon seeing my home wifi network will automatically connect. The first raspberry pi will be pinging every minute a list of statically defined addresses looking for a phone or laptop, mine or my wife’s.

Once it receives a response, it sends a WOL packet that will turn my file server on. Now that it is on, the first raspberry pi will only keep pinging the file server, as long as it is on, there is no need for further checks.

On the file server, it would have a script running, pinging another list of addresses. As long as one of them answers, it will do nothing. When all addresses on the list fail to answer on the past 30 minutes it will initiate a shutdown.

And the cycle repeats.

When there is someone at home, the file server will be on, when everyone leaves, it turn itself gracefully off.

Perfect!

Since I have only ONE raspberry pi to do this, I will be turning my file server manually on for the time being and focusing this post on the second part.

This is the script I am using on my FreeNAS box:

#!/bin/sh

CHECK_EVERY=60
MAX_FAIL_COUNT=30

keep_on() {
  for p in htpc.home raphael-pc.home sala-tv.home teste.home;
  do
    if ping -c 1 $p >/dev/null 2>&1; then
      return 0
    fi
  done
  return 1
}

# Client must be up before starting main loop
while sleep 5
do
  if keep_on; then
    break
  fi
done

FAIL_COUNT=0

# main script
while sleep ${CHECK_EVERY}
do
  if keep_on; then
    FAIL_COUNT=0
  else
    FAIL_COUNT=$((FAIL_COUNT+1))
    echo $FAIL_COUNT
  fi
  if [ $FAIL_COUNT == $MAX_FAIL_COUNT ]; then
    shutdown -p now
    exit
  fi
done 2>&1

Not much complex stuff. The variable CHECK_EVERY state that the checks should be every 60 seconds and MAX_FAIL_COUNT that after 30 fail attempts it will shut itself down.

There is one failsafe: The script will only act when it receives an answer from at least one device on the list. This is to prevent the box to be turning off if something goes wrong with my internal DNS or if I plug it on someone else’s network. You never know…

To allow this to persist between boots, I first made the root writable with

su
mount -uw /

Then, I saved this script on /conf/base/etc/autoshutdown.sh and added a line calling it on /conf/base/etc/rc.local:

#!/bin/sh

/conf/base/etc/autoshutdown.sh

Also, made both scripts executable.

And that is it!

When I get a hold of a second raspberry pi I’ll post the other scripts here as well.

Letting your pool sleep…

Some very good points on an article I just stumbled upon…

Mount all your filesystems/pools with noatime

This way you won’t generate writes every time a file is accessed. I had this suggested by an episode of TechSnap where one of the hosts mentioned that they do this to avoid writes while doing reads but never came back to actually implement it.

I don’t have other filesystems on my FreeNAS box and ZFS has a property for this. Just run:

zfs set atime=off POOLNAME

Find files modified in the last day or so

A good snippet to try to get to these files is:

find / -mtime -1

Relocate directories and files to non-rotating media

Also another great suggestion on the original article:

Get a cheap USB drive (does not need to be big) and format it as ext4 (technically, you could set up another ZFS pool there too). Then, set it to be mounted in `/var/volatile` on your fstab. You can now move directories that contain frequently modified files there. After you’re done moving those directories, you can symlink them from their original location. So, for example, you would move `/var/log` to `/var/volatile/log`, then creating a symbolic link to `/var/volatile/log` named `/var/log`. At this point, it would be wise to make a cron job to nightly back the contents of this USB drive up (think `rsync -a`) to a backups directory somewhere in your pool. OK. If you’ve moved the most frequently modified files to `/var/volatile`, your disks will be idle unless you are actually using your file server. Now it’s time to take advantage of that idleness.

Problems when Windows and Macs access the same FreeNAS shares

I believe that this might be quite a common issue among other FreeNAS users. I have in my network both Mac and Windows PCs and after I mount a share in OS X, multiple folders are created, namely “Network Trash Folder” and “Temporary Items”.

After trying to ignore them for quite a long time I decided to take a deeper look into this. The Oreilly’s Samba Book, Chapter 5, section 2 details some options that can be used here.

The Ubuntu forums also mention this solution which ended up being simpler than I could have expected. I just added the following line to CIFS Settings on my FreeNAS box:

hide files = /Network Trash Folder/Temporary Items/

That´s it!

“Warning: /var/lib/mlocate/daily.lock present”

I am still fixing small issues as they appear in my home setup. Right now I have a file server running FreeNAS named capella.home and a virtualization box running Proxmox named andromeda.home.

I configured andromeda to map a NFS share from capella as a repository for images and templates and for backups to be saved as well. Everyday andromeda performs full backups of all my VMs but to conserve power and to preserve the hardware I turn capella off whenever I’m travelling and every single time I was getting multiple warning e-mails with the message:

/etc/cron.daily/mlocate:
Warning: /var/lib/mlocate/daily.lock present, not running updatedb.
run-parts: /etc/cron.daily/mlocate exited with return code 1

This message was being sent both from andromeda and multiple VMs hosted inside it.

At first I thought that this was somehow due to auto-upgrade issues, even scheduled my VM host to auto reboot every couple days to see if it would avoid it (terribly bad practice, I know) with no success.

It turns out this was happening because the NFS server was offline and mlocate was trying to index it, so I adapted my puppet base recipe to include the following:

#
# locate, mlocate and updatedb
#

# Limit where updatedb scans
file {'/etc/updatedb.conf':
  ensure  => 'present',
  owner   => 'root',
  group   => 'root',
  source  => 'puppet:///etc/base/updatedb.conf'
}

and borrowed the contents of updatedb.conf from :

PRUNE_BIND_MOUNTS="yes"
PRUNENAMES=".git .bzr .hg .svn"
PRUNEPATHS="/tmp /var/spool /media"
PRUNEFS="NFS nfs nfs4 rpc_pipefs afs binfmt_misc proc smbfs autofs iso9660 ncpfs coda devpts ftpfs devfs mfs shfs sysfs cifs lustre_lite tmpfs usbfs udf fuse.glusterfs fuse.sshfs ecryptfs fusesmb devtmpfs"

And that is it!

Setting Debian time zone with puppet

To manually change the time zone on a Debian install you would naturally use the interactive command dpkg-reconfigure tzdata.

In order to change all your puppet managed machines at once I use this simple recipe:

#
# Timezone -> America/Sao_Paulo
#

package {'tzdata':
  ensure  => 'present'
}

file {'/etc/localtime':
  require => Package['tzdata'],
  source  => 'file:///usr/share/zoneinfo/America/Sao_Paulo',
  notify  => Exec['reboot']
}

file {'/etc/timezone':
  content => 'America/Sao_Paulo',
}

That’s it!

Local DNS resolution for in-house devices

2013-03-20 23.16.05

Three great articles on how to setup my DD-WRT router to take advantage of Local DNS resolution. No more typing IP addresses going forward, everything is ipad.home or server1.office.

As an added bonus I also modified the resolution of domains related to advertising to an invalid IP, less junk! Great!