ProxmoxVE 4.x OpenVZ to LXC migration

At Onlime Webhosting we chose ProxmoxVE as our favorite virtualization platform and are running a bunch of OpenVZ containers for many years now, with almost zero issues. We very much welcome the small overhead and simplicity of container based virtualization and wouldn’t want to move to anything else. ProxmoxVE added ZFS support by integrating ZFSonLinux back in Feb 2015 with the great ProxmoxVE 3.4 release – which actually would have deserved to bump its major version because of this killer feature.

Previously we have been running our OpenVZ containers on plain Debian Linux boxes. But with Debian Wheezy (7.0) the Debian community decided to no longer include a kernel which has been patched with the OpenVZ extensions. Switching to ProxmoxVE at that time was a piece of cake as ProxmoxVE was just a plain Debian Linux with a RHEL kernel, OpenVZ and KVM support, and some nice web interface to manage the containers and VMs. Actually, we never really needed the ProxmoxVE web interface (GUIs always suck!), but that one is quite lightweight and very well integrated. If you don’t like it, use the provided CLI tools – ProxmoxVE does not force you to use the web interface at all.

In 2014 I did a review for Proxmox High Availability (by PACKT publishing). But at Onlime, we never employed any cluster technologies to date. I was always looking for simple solutions without unnecessary complexity. ProxmoxVE with OpenVZ on ZFS pools was the perfect match. We are replicating our containers from one host node to another every 10mins with simple and super fast zfs send|receive via a modified version of ZREP.

In ProxmoxVE 4, OpenVZ support was removed in favor of LXC. Both, KVM and LXC are built into any newer Linux kernel, so in the long term (and due to the nature of OpenVZ as a huge kernel patch which is hard to maintain) it was clear we had to give up OpenVZ. Even though the OpenVZ developers are currently focusing on merging the OpenVZ and Virtuozzo source codebase and have just released OpenVZ 7.0 I don’t give OpenVZ a long future any more. In comparison to LXC, the tools for OpenVZ are still much more mature and powerful, but lately LXC made a lot of progress and having it perfectly integrated into ProxmoxVE is a great step forward. So, we have decided to migrate to LXC.

(Sorry about the long preface. You probably don’t care about history reading this blog post. I am getting to the point now…)

What I cannot believe and what prevented me to look into LXC on ProxmoxVE up to now is the clumsy migration path that ProxmoxVE suggests on ProxmoxVE Wiki: Convert OpenVZ to LXC. Basically they suggest to dump and restore a full container as follows:

Srsly? Do you guys only have containers with sizes below 1GB and don’t care about downtime at all? Real world looks different and I would not even go this migration path with a 5GB container. I don’t even want to think about those containers with ~ 1TB of data.

If you are running OpenVZ containers on ProxmoxVE, sure you have some easy solution to migrate your containers from one host node to another. As I said, we are using ZREP for container replication and have written a wrapper script that migrates a container to its failover host node. That’s what I suggest and please forget about live migration – it never really was working in OpenVZ (well, it was, but I am talking about a really stable solution that never fails) and LXC lacks any kind of hot/live migration. Doing a ZREP presync, stopping the container on the source host node, doing the main ZREP sync, finally starting the container on the destination host node is just a matter of seconds, usually causes a downtime of 10s up to 30s max (on a larger container).

Read More

Setting up PHP & MySQL on OS X Yosemite


Homebrew is a package manager for OS X. Install it, as we’ll need it later:

You’ll also need the Xcode command line tools – at least version 6.1. Xcode is available on the Mac App Store as a free download. Don’t forget to start up Xcode at least once after initial installation.

Initially and after every major OS X upgrade, you might need to reinstall the Xcode command line tools and accept the license aggreement:

Also, don’t forget to create this missing symlink after upgrading to a new major OSX version:

Once Homebrew is set up and you want to upgrade outdated packages, run:

Remember to also run brew update && brew upgrade after upgrading to a new major OS X version.

List all installed packages:

You should never be required to run brew as root using sudo!

Read More

sendmail-wrapper for PHP

Years ago I wrote about the extensive sendmail wrapper with sender throttling, a pretty simple Perl script. It reliably provided throttling of the email volume per day by the sender’s original UID (user id). It also logged the pathes of scripts that sent emails directly via sendmail (e.g. via PHP’s mail() function). The main flaw in the original sendmail wrapper was security, though. As in Linux, every executable script must be readable by the user that calls it, the throttle table in MySQL was basically open and every customer could manipulate it. Every customer could raise his own throttling limit and circumvent it.

Today, I’m publishing my new sendmail-wrapper that is going to fix all the flaws of the previous version and add some nice extras.
The new sendmail-wrapper is written entirely in PHP and does not require any external libraries. It is a complete rewrite and has pretty much nothing in common with the old Perl version.

This project is hosted on Github


  • Lets you monitor any mail traffic from PHP scripts
  • Allows throttling (limiting) emails sent by PHP’s mail() function
  • Throttle by sent email and/or recipient count per day
  • Logs both to syslog and database with message metadata
  • Logs common mail headers like From, To, Cc, Bcc, Subject
  • Fixes Return-Path header on the fly for users who did not correctly set it
  • Highly secured setup, customers cannot access the logging/throttling database
  • Standalone PHP application without any external library dependencies
  • Built for shared webhosting environment where PHP runs as cgi/FastCGI/suPHP
  • No cronjobs required, sendmail-wrapper will reset counters automatically every day

Read More


Webhosting customers are messies, at least some of them – or (sadly, that’s the truth) the bigger part of them. Some people still think they can run the same blog software or CMS for years without ever caring about upgrading. I tell my customers over and over how important it is, to keep their website up-to-date and don’t let any outdated code lying around. Still, as long as their website doesn’t get hacked or defaced, they don’t really seem to care.

If you’re in the same situation as me and you are providing webhosting services to your friends or customers, read on.

This project is hosted on Github

Read More

Process hiding: hidepid capabilities of procfs

Five years ago I wrote about kernel based process hiding in Linux (see articles Simple process hiding kernel patch, Process hiding Kernel patch for 2.6.24.x, RSBAC – Kernel based process hiding). It got time to continue the story and finally present you a real solution without the hassle of a self-compiled kernel.

How can I prevent users from seeing processes that do not belong to them?

In January 2012, Vasiliy Kulikov came up with a kernel patch that solved the problem nicely by adding a hidepid mount option for procfs. The patch landed in Linux kernel 3.3.

In the meantime, this patch luckily also landed in the 3.2 kernel of Debian Wheezy (see backport request in Debian bug report #669028). This feature has been also pushed back into the kernel of Red Hat Enterprise Linux 6.3 (see RHEL 6.3 Release Notes), and from there to CentOS 6.3 and Scientific Linux 6.3. Recently, this feature was even backported to the 2.6.18 kernel in RHEL 5.9.

As Proxmox VE currently runs on a RHEL based 2.6.32 kernel, it’s also supported in my favorite OpenVZ/KVM virtualization platform. Great!

Read More

Proxmox VE: Restricting Web UI access

With the release of Proxmox VE 3.0 back in May 2013, the Proxmox VE web interface does no longer require Apache. Instead, they’re using now a new event driven API server called pveproxy. That was actually a great step ahead, as we all know Apache get’s bulkier every day and the new pveproxy is a much more lightweight solution. But the question arose:

How do I protect my Proxmox VE WebUI with basic user authentication?


Basically, we do not trust any web application out there so we better double protect the whole WebUI with plain old basic auth – previously done in Apache by .htaccess.

The main idea:

  • Restrict access to the pveproxy (= Web UI) to localhost
  • Install a local Nginx web proxy server that forwards requests from port 443 to pveproxy’s port 8006 and restrict access to it using HTTP BASIC AUTH

Read More

Fix Snow Leopard issues with SMB

Apple’s OS X Leopard (10.5) had problems with accessing Samba shares. Most of them got fixed in late Leopard releases, 10.5.4/10.5.5. During the last Leopard releases the hope arose, that finally OS X will make it into office environments sharing Windows networks.

Then, Apple released Snow Leopard (10.6). Another step into the right direction by providing built-in support for Exchange Server and Google services. But what happened with SMB? The whole story started all over again. While Samba-shares worked pretty well on Leopard, in Snow Leopard all seemed to be messed up again.

Read More

Porting Agavi’s shiny error handler to Zend Framework

Agavi-logoLast Tuesday I joined the Webtuesday Zurich meeting. David Zülke (project lead, Munich) gave a talk on Agavi Framework.

Agavi is a general purpose PHP application framework built around the Model-View-Controller architecture originally based on the Mojavi 3 Web application framework written by Sean Kerr. It provides a rich toolset that solves most of the routine problems in Web application development. (…)

There are some great ideas and programming techniques behind Agavi and a very bright engineering team. Congrats, David, that was a great presentation of a great project! Agavi is not yet well known in PHP community but you definitely should give it a try! There’s a well though-out architecture behind it.
There will be a webcast of David’s presentation available soon on Webtuesday Zurich.

Read More

EncFSVault as a FileVault replacement

EncFSVault provides a replacement for Apple’s FileVault. There are a lot of issues with FileVault. Personally I don’t like any proprietary software for security sensitive storage of my data. But the main reason I was not able to use FileVault was the fact that FileVault still doesn’t provide support for case sensitive HFS+ file systems as of OS X Leopard 10.5.6. That’s a shame!
My choice was EncFSVault. Good or bad choice?

Read More

More OS X Leopard Tips & Tricks

I collected some more Tips & Tricks to tweak OS X Leopard (also see my previous article). In my eyes, these modifications are all 100% necessary for a great daily Mac experience. I was a Windows guy and I’m running Debian/Ubuntu Linux for the last 7 years for server purposes. But now, for daily work I really start to get used to my MacBook Pro running OS X Leopard 10.5.6. I still hate Apple for lately focusing on its schicky-micky customers with all their shiny glossy expensive housewife gadgets. But I love OS X. It just needs some tiny little tweaks…

Read More

Extensive sendmail wrapper with sender throttling

In this tutorial I’d like to describe how to create an extensive sendmail wrapper for a web server to monitor all sent emails and throttle daily sent email volume by the senders original UID (user id). This is useful if you e.g. run PHP in CGI-mode with SuExec, that is: all customers run their scripts under their own UID. The wrapper described here is not just a PHP-only wrapper (as described in my Simple PHP mail wrapper tutorial) – it directly replaces /usr/sbin/sendmail so we are able track all sent email of the whole system.

Read More