You’re already asking the right question: “most physically demanding” depends on what variable you’re measuring. Historically, militaries have always produced “hard men” through deprivation, load, and pace—what changes is how systematically it’s done.
If we define physical demand as sustained load carriage plus endurance under uncertainty, the lineage is older than modern SOF. Think of Roman marches (vegetius describes weight and pace expectations), then fast forward to WWII raiding forces (Commandos, LRDG, SAS origins) where selection and training emphasized long navigation marches and operating with minimal support. During the Cold War, doctrine split: some units optimized for long-range reconnaissance and unconventional warfare (months-long deployments, heavier rucks, sustained patrolling), others for maritime/direct action (water confidence, finning, cold exposure), others for hostage rescue (short, explosive bursts with high skill and repetition).
So: certain selections become famous (and rightly so) because they compress hardship into a short window. But “hardest” overall can look different across a career: a unit with brutal selection might have a different steady-state training burden than one with constant workups, schools, and deployment cycles. In my view, the most physically demanding “unit” is often the one whose mission forces constant long-duration movement with load (and little rest), rather than the one with the most cinematic selection week.
For grounding, look at how each unit’s doctrine shapes conditioning: endurance + load + navigation (LRRP/recce), water + hypothermia management (maritime), or short-duration maximal output (CT).