Maintenance window Wednesday 2017-09-06" -- FINISHED
Monthly maintenance window begins at 0900 hours on the first Wednesday of the month. (That is today.)
This time we will:
- Upgrade Linux kernel and other system software on Bianca, Castor, Fysast1, Grus, Irma, Milou, and Rackham.
- Stop Slurm queues on Bianca, but not on the other systems.
- Restart login nodes on all systems, probably once only.
- Begin to test an alternative way to mount Pica on Milou nodes m[191-208].
Most of the day, logins to Bianca will be unavailable.
Part of the day, Grus will be unavailable.
Compute nodes on Fysast1, Irma, Milou, and Rackham will be upgraded between jobs, so Slurm queue waiting time may be somewhat longer today.
We plan to keep you informed about our progress with the maintenance, by updating this page approximately every three hours.
Update at 1100 hours
Bianca is still unavailable.
Maintenance is progressing. Very soon all login nodes on Irma will be restarted. We have restarted login nodes on Fysast1 and Milou and will now verify them.
Update at 1440 hours
Bianca is still unavailable, and its maintenance takes longer time than expected. Perhaps we will not finish until the middle of the night.
We have finished maintenance on Castor, Fysast1, Grus, Irma, Lupus, Milou, and Rackham.
Update Thursday 0200 hours
Unfortunately a problem remains on the Bianca cluster which will need further investigation starting later this morning. It is expected at this time that Bianca will remain offline during the first half of today (Thursday). Please continue checking this page for regular updates.
Update Thursday 1100 hours
Work is progressing. After lunch we will likely be able to provide a better estimate when Bianca will be online again.
Update Thursday 1400 hours
Unfortunately a few issues still remain to be solved.
Update Thursday 1630 hours
We currently expect to bring the system back online during Thursday evening assuming no new issues are discovered.
Update Friday 0000 hours
Bianca should be available and jobs are running. Some minor details are still to be done, but you should be able to access Bianca again.
Update Friday 1300 hours
September maintenance completed.
Maintenance window Wednesday 2017-09-06" -- FINISHED
milou2 rebooted August 28
milou2 rebooted Monday 2017-08-28 at 19:51.
Replacing (nearly) all disks on Irma's compute nodes
We're restarting irma-q for technical reasons. The slurm queue system may be unavailable for submitting/verifying job status for a few minutes.
milou2 rebooted August 19
milou2 rebooted on Saturday 2017-08-19.
Bianca's storage system Castor had a hiccup yesterday Thursday -- FIXED
Maintenance window Wednesday 2017-08-02 -- FINISHED
Unexpected reboot of Pica at Monday morning.
Restart of two Milou login servers today Thursday
Lower service level during UPPMAX holidays
Part of storage system Pica is still very slow
Pica was partly restarted just now, please look for problems in your job output
UPPMAX had to restart part of storage system Pica, because it worked too slowly with nearly no read/write traffic.
The restart was done a little after 1300 hours.
For Rackham users, this meant that you might have had problems with reading and writing to your home directory.
For Milou users, this meant that you also might have had problems with reading and writing to your home directory. But for Milou users, also reading from /sw (where the modules live) and reading and writing to some project directories were affected.
Please look one extra time for problems in your job output, for jobs running at this time.
We are sorry for the inconvenience.
On Milou and Rackham, very difficult to login or otherwise use /home directories -- FIXED
UPPMAX has problem with an extremely slow access to /sw (where e.g. modules live) and home directories on Milou, and to home directories on Rackham.
Because of that, it is very difficult to login to Milou and Rackham.
We will investigate the source of this problem, and will report any success as updates here.
Update at 1310 hours
We restarted part of Pica, and that solved the problem
Hopefully your jobs will continue without problems, but please be careful and look once extra time for errors in your job output.
SUPR and C3SE website down
SUPR and C3SE websites are down at the moment. This prevents you from using SUPR at the moment. Please try again later
No maintenance planned for today's maintenance window
First (non-holiday) Wednesday of each month is UPPMAX's normal, planned maintenance window.
But today we will do no maintenance.
Next maintenance window is 2nd of August.
Restart of login server milou-f Tuesday morning -- FINISHED
File system mounts of Pica volumes was not working correctly.
This was fixed by a restart of the server. Now it works much better.
We are sorry about any inconvenience for you due to this.
Lost contact with Milou nodes m[1-48] for an hour this morning -- FIXED
From approximately 0800 hours to 0910 hours this morning, an ethernet switch in Milou lost power, making 48 nodes unavailable.
Two jobs got NODE_FAIL when trying to start, and interactive work on these nodes was denied. Otherwise, we seem to have had no problems with the temporary network loss.
Singularity is available
Urgent kernel upgrade -- FINISHED
Today we are performing an urgent kernel upgrade on Milou, Fysast1, Rackham, Irma, and Bianca. Login nodes will be restarted during the day. No running jorbs or queues are stopped. We will update on the progress here in System News during the day.
UPDATE 16:00 - Update completed.
Intelmpi performance issues
Bianca graphical login now working
Uses Thinlinc Web Access. Not X-forwardning.
Bianca's storage system Castor has problems -- FIXED
Maintenance window Wednesday 2017-06-07 -- FINISHED