http://github.com/hinoglu/Domo/ is a python based remotely manageable web crawler based on Python, multiprocessing, pyro, lxml and pycurl.

It's main purpose is to be able provide a lightweight python based alternative to archive.org's heritrix. Main structure is similar to heritrix and Domo provides almost similar but somewhat lesser features.

Though Domo is a web crawler/archiver, it can be used for anything that requires remote multiinstance / multiprocess job management features, thanks to Pyro and multiprocessing libraries.

I'd be glad if anyone would like to contribute or join the project.