distributed computing

  • guess you have a bunch (4000) of embarrassingly parallel tasks (one hour each) and access to about 40 CPUs through SSH. All tasks would run easily on each node, and they all share some network drive (NFS). Would be nice to run everything from just one place (script, command-line, web interface, ...)

a bunch of existing tools

what we can do