David Chisnall (*Now with 50% more sarcasm!*) (@david_chisnall@infosec.exchange)
@halcy@icosahedron.website @whitequark@treehouse.systems @hayley@social.applied-langua.ge @casandro@f-ckendehoelle.de We also hit something like this with CPU affinity work in FreeBSD. Adding CPU affinity in a 24-core build machine made builds take longer. It turned out that, on modern systems, snopping a peer's L1 cache is *really* fast, so the cost of a process migrating between cores was very low. But each job was sitting in hot loops that heated up specific bits of the cores. Thermal throttling kicked in for the core (or core cluster) triggered by the hottest bit. Semi-random shuffling of jobs around heated the chip more evenly and made the thermal throttling kick in later. This was even more noticeable in the less parallisable parts of the workload because it also meant making each core idle in turn for a bit, which uniformly improved cooling.
Infosec Exchange