Open source project: Func, the Fedora Unified Network Controller

Labels: Posted by EdGe on Thursday, February 21, 2008 at 7:57 PM

Author :Michael DeHaan


Func had an interesting beginning. It began not in a whiteboard-lined conference room, but in a small coffeeshop in Chapel Hill, North Carolina. Greg DeKoenigsberg, Adrian Likins, Seth Vidal, and I were discussing how to make Linux easier to manage for large install bases. That’s when we came up with the idea for Func.

While Fedora contains excellent open source management applications for a variety of tasks, it still lacked a good remote scripting framework roughly analogous to the features provided by system-config-* applications. It turns out this was something many of us wanted to write for a long time–but for some reason, we never did. So, why not build it?

A fair amount of commercial management software seems to get built and sold without consulting the people who end up using it–systems administrators. While these applications may present extremely well-crafted graphical user interfaces with enterprise-grade reliability and scalability features, they often lack solid scripting ability or require development using complex SOAP APIs to get things done.

For managing very large install bases, these aspects impose barriers to automation. System administrators tend to prefer things written in Perl, Python, or bash. Automation is critical.

The most commonly used remote management tool for Linux is probably SSH. While being a very useful tool for manipulating a single machine remotely, it is challenging to integrate with an environment where machines are frequently reinstalled or where complex remote actions need to be scripted. SSH wasn’t meant to be a multi-system remote scripting tool, and it’s definitely not meant to be something you build other applications on top of. Futhermore, integrating SSH key deployment with kickstart (even with tools like Cobbler to help) can be difficult.

On the other end of the management spectrum, there are configuration management systems such as Puppet, cfengine, and bcfg2. These solutions are great for pushing configuration files around and describing the way infrastructure should look (or making it look that way), but are not as well-suited for remote scripting and one-off tasks.

We wanted to create a solution that filled this void–something absolutely simple, rapid to deploy, easy to use and easy to expand. This would become Func.

Furthermore, we wanted to challenge ourselves, so we decided to create the first release of Func in two weeks time. This was a goal we managed to exceed, as we had it submitted to Fedora in about eight days.

Func works by having a very minimalistic daemon (funcd) installed on each managed machine, which we call a “minion.” Each minion, when it is first run, receives SSL certificates from a remote “certmaster,” which can either be automatically signed or manually approved by an administrator. Client software (in the form of the command line tool (“func”) or the Client API) can then address specific minions from the central server (called the “overlord”), or even address a large set of them at once. Communication is currently only from the overlord to the minion, but intra-minion communication is coming.

To help describe what func can do, the following command shows the available system memory on all example.org machines being managed.

func “*.example.org” show hardware systemMemory

The above also illustrates Func’s globbing feature. Similar globs, such as “*” or “a*” work as expected–communicating with all servers, or all servers starting with “a”, respectively. Of course, addressing only a single system works as well.

The Func project page also lists example code for doing the same thing (for various func modules) in just a handful of lines of Python. This should be easily understandable even if you do not know Python. (And if you don’t, it’s easy to pick up.)

Here’s a quick Python example:

import func.overlord.client as fc
client = fc.Client("*.example.org;*.example.com")
client.service.start("acme-server")

The initial Func release contained modules for remotely manipulating services, viewing hardware inventory (via Smolt), running remote commands, and many other tasks commonly found in systems management apps. More importantly though, it exposed a trivially simple pluggable model, allowing any application to drop in a module on a remote machine and instantly have it be accessible by the Func “overlord”, whether by command-line or Python scripting. Func is not strictly for systems management–Func is a truly pluggable framework for any application that needs two-way secure communication.

An example of Func’s power is shown by the func-inventory application. Func-inventory is an application that checks on all of the nodes in your infrastructure, and inventories all the Func modules they have running. The results are stored in git (a distributed version control system), and can be viewed with apps like “gitk,” “gitweb,” or “git log.” Func-inventory can therefore be used to see if drives disappear, or if new packages are installed. It is very easy to use Func-inventory to report on all types of changes throughout an organization.

While this is interesting, it is more impressive to note that Func-inventory is only about 200 lines of Python, and was written in only half of a work day. Func contains a very powerful scripting API. Func-inventory ships as part of Func and is installed into /usr/bin.

Other applications contained in Func’s source tree as examples include an exploding battery finder for laptops (which would have been very handy earlier this year) and a failed drive detector (that works by using SMART). Each of these applications are really only a handful of lines of Python. If you’re a Perl or bash hacker, Python is very easy to pick up and Func may get you hooked.

Another useful feature of Func is newly added support for parallelism. Func operations running on remote machines may be slow to complete. They can now be executed in multiple processes, with Func handling the multi-process aspects and combining results as if things were executed in a single process. This is supported both via the Func command line and the Python API. More performance-related tweaks will go into Func as time goes on.

Func is still young. Since starting the project only a few months ago, interest in Func has grown rapidly. It has a IRC channel (#func) on irc.freenode.net, as well as a mailing list. We’ve received a wide variety of patches, and are happy to see the beginnings of support for other distributions, with contributions including both BSD and OpenSuSE. The great advantage to open source is in being able to collaborate with such a diverse user base. Whether you have an idea for a new module, need a secure network communication path for your new application, or just want to use existing Func modules to automate your environment, everyone is invited to stop by IRC and the mailing list.

Want to install Func and try it out? Func is available in Fedora and EPEL. See the Func project page for more details.

We would like to reiterate that Func is your application–by sharing ideas and features among its users, Func grows more powerful for everyone that uses it–the true beauty of Open Source. If you write an interesting Func module, we hope you’ll share it with us. Func modules are easy to write and we expect to amass a very large library of them.

If you have a need to manage a very large number of remote machines and are wish for something a bit more sophisticated than SSH for automation purposes–or just need a secure remote communications channel for a new project–Func is the application for you.

Resources

Source : Func