Personal tools
You are here: Home Writing My HOWTO Zone Gentoo Clustering

Gentoo Clustering

This document describes how to maintain packages on a Gentoo cluster with a focus on good distributed design principles.

Distributed computing emphasizes transparency, consistency, and reliability among other features.

This article describes how to manage a cluster (or LAN) of identical machines effectively. We propose a tools-based approach.

Here's how I do things:

  1. Make sure you set up SSH with public key authentication. It's a big pain to manage clusters and type in passwords all the time. If you are loathe to doing so at least consider host equivalence of all nodes within your cluster so you can log in password free once past your head node of the cluster.
  2. Make sure you have unison running on every computer within the cluster.
  3. Always build a binary package when running emerge. The --buildpkg option is recommended. For your own sanity, consider only building packages on one of the nodes at any given time.
  4. Note: Although you are building packages on one node, this does not mean that you are limited to doing so. Because packages and Portage metadata are kept coherent in our approach, you can use any node as a master should your master node fail unexpectedly!
  5. Perform one way synchronization from the head node using a tool such as rsync or (better yet) Unison of the /usr/portage/packages/All directory. You can do this to each/every node. This can be automated with a script.
  6. Use ssh to perform a remote emerge -k system. The -k is essential to ensure the binary packages you've replicated are actually used.
  7. Do the etc-update on the head node only. Unfortunately, etc-update still requires some manual intervention at times. Nevertheless, you only want to be doing this in one place and then synchronize as before using unison.

That is the approach in a nutshell.

Document Actions