Skip to main content

Compress files fast and efficiently with pbzip2

Printer-friendly version

Question

How can I compress my files as fast and efficiently as possible, in order to save space?

Answer

You can use this SBATCH script [1] to run the compression in parallel as a node job, with a parallel version of the highly efficient bzip2 compression software.

Remember to modify the appropriate #SBATCH parameters at the top of the file, according to your project, and the estimated time to compress your files.

[1] Thanks Roman Valls Guimera, for contributing this script.