Compress files fast and efficiently with pbzip2
Question
How can I compress my files as fast and efficiently as possible, in order to save space?
Answer
You can use this SBATCH script [1] to run the compression in parallel as a node job, with a parallel version of the highly efficient bzip2 compression software.
Remember to modify the appropriate #SBATCH parameters at the top of the file, according to your project, and the estimated time to compress your files.
[1] Thanks Roman Valls Guimera, for contributing this script.
Submitted by Samuel Lampa on Tue, 2011-09-13 15:32
