At the Center for High Throughput Computing at UW – Madison, we don’t install software system-wide for specific researchers, except for common, well-used system libraries. Instead, we recommend that individual researchers download or build a copy of their desired software locally. In this post, I’ll talk about why we do this, how it works, and how to support this particular approach.
Why handle software this way?
Our first reason for handling software this way is both practical and philosophical. Software has to be installed somehow — we could either spend staff time installing software or we could spend staff time training people to install their own software. We have opted towards the second option because of our philosophy that it’s better to give researchers the freedom (and responsibility) to manage their own software. Practically speaking, we also think that this researcher-driven approach is ultimately more scalable in terms of staff support time and allows researchers to use the software they want without waiting for a staff member to have time available to install it.
The second reason for this approach is that it empowers researchers to run at a large scale more easily. In our high throughput system, the fact that we start researchers with portable, personal software installations facilitates their leap from running solely in CHTC to using other grid resources (where having a system software installation isn’t an option).
How does this work, actually?
Many scientific programs are built to be portable. The easiest scenario is when a program is available as a single binary executable (or a folder of them). As one example, well-developed bioinformatics tools (blast, bowtie, bwa) fall in this category.
Other programs are installable to a single directory, and that directory can be copied from computer to computer and used in the new location. One example is Python.
The final level of challenge is the need to sometimes install and bring along not only the main software package, but a few dependencies.
Some programs are more challenging to install than others. In our “do-it-yourself”, portable approach, software with a lot of dependencies can be challenging to install. Another major problem is when software cannot be installed locally, or hard-codes its installation location (preventing it from being used elsewhere).
One approach that we’ve started to use for these difficult programs is containers. Because containers provide a consistent, self-contained environment for installation, they are a good solution for tricky software. Our high throughput system scheduler (HTCondor) can run jobs using Docker containers and other resources can use Singularity.
What’s required to support users?
Most of our user support for software installations happens inside our usual
user support structure:
- We meet personally with each researcher who gets an account at CHTC, where we talk through the rationale and process for installing software.
- We have user guides for common software installation processes (see our guides for Matlab, Python, and R).
- Researchers can ask follow-up questions via email or during office hours.
Follow-up questions typically fall into two camps:
- someone who is doing a fairly standard installation and just needs some extra help
- someone who is installing something we’ve never seen before and it’s weird
It’s the last set of questions that is the most time-consuming. However, we usually don’t have more than 1-2 of these per month and are also able to draw on the expertise of colleagues to find solutions. Overall, we don’t spend a lot of time having to work on software installation issues, compared to other problems that crop up.
Recommendations for software developers
There is plenty that scientific software developers can do to make their programs easy to run on large-scale systems in the ways I’ve described. My wishlist would include:
- ability to run/install software without sudo or other administrative privileges.
- not using hard-coded paths to a persistent location; having flexibility in where software is installed vs run and where input and output files or directories can be made.
- using environment variables or configure files to manage location information.
- for tools with lots of dependencies, providing a software development kit (SDK) or other kind of dependency manager as a pre-requisite to installing the software.