Building Our SC25 Cluster at E4 in Italy

The next step on our road to SC25 started early on a Sunday morning, as three of our team members packed their bags (and some snacks) and headed south – destination: Rubiera, Italy, home of our hardware sponsor E4 Computer Engineering. After a few hours on the train, we arrived just in time for dinner and a first taste of the Emilia-Romagna cuisine – the perfect fuel for the days ahead.

On monday morning, Hussein (our mentor from CSCS, and resident HPC oracle) picked us up and drove us to E4's headquarters. There, Davide from E4 welcomed us and showed us around before we got started.

Right away, the cluster-building spirit kicked in – cables, racks, and nodes were quickly in place. Of course, it wouldn't be an HPC build without a few hiccups: our USB stick turned out to be broken, and the SSD refused to be recognized at first. After some troubleshooting (and a few rounds of coffee), we still made solid progress – we even managed to run HPL on two nodes before the end of the day!

For lunch, we enjoyed a proper Italian meal nearby – clearly, productivity scales with pasta. Dinner was another highlight: we went out with Chicco, who turned out to be not only an expert in high-performance computing but also in high-performance dining.

Tuesday was dedicated to debugging. While the cluster was already up and running, our NVIDIA drivers decided to test our patience. Most of the day was spent fixing and reinstalling them, with Hussein and Davide helping us navigate compatibility issues and kernel quirks.

Despite the driver drama, we made good progress and learned a ton about debugging HPC software stacks in real-world conditions. Lunch and dinner were, once again, pure Italian bliss – this time ending the day at a cozy restaurant inside a restored castle.

Wednesday came with a new challenge: InfiniBand drivers and the OFED stack. Together with Hussein and Davide, we dug deep into system logs, rebuilt drivers, and worked on getting the NVIDIA and InfiniBand stacks to coexist peacefully. By lunchtime, most of the major compatibility issues were resolved – a satisfying milestone to close out the trip.

After a final round of espresso, we packed up and headed back to Zurich – tired, happy, and with a cluster that's well on its way to powering our workloads at SC25 in St. Louis.

A huge thank you to E4, Davide, Chicco, and Hussein for their expertise, patience, and warm hospitality. The build may have tested our debugging skills, but it also reminded us why we love this work – the perfect mix of problem-solving, collaboration, and pasta.