Considering both loss and power handling, systems with frequencies above 300 MHz typically use N-to-N (or N-to-UHF) configurations. In real-world engineering applications, N-to-N configurations are more common than N-to-M configurations. The reason for this is likely due to historical trends. However, when looking at professional Hytera and Motorola systems operating in the 400 MHz band, there is very little use of M-to-M configurations; only amateur radio systems utilize them. Similarly, Hytera's VHF repeaters also use N-to-N configurations.
Furthermore, considering that BNC connectors have lower connection stability than N connectors, it is acceptable to use M connectors for HF and 50 MHz (some specialized shortwave devices also use N connectors). If you prioritize low loss in the VHF range, N connectors are also suitable. For frequencies above 300 MHz, I recommend using N connectors. I have already replaced all my local repeaters with N-connector systems.
Regarding SMA, it is rarely used in high-power systems, except for jumpers and high-frequency low-power transmission before the amplifier.