Look up “Beau Brummely” and “Great Renunciation”.
US southern-prep khakis, blazer, shirt and tie, slightly longer hair is pretty much the exact look Brummely and his dandies created and socially enforced through mockery.
Some people claim that the underlying force granting Brummely the ability to do so had to do with the center of power shifting from traditional nobility to financiers.
Dude on the left pulls it off, dude on the right does not, imo.