My ex did exactly this. If we went on a long car journey, he’d almost always want to drive ( he always did, as he liked being in control), but then when he got tired, he’d ask me to take over, which I happily did.
Apart from criticising my driving, he would unnerve me by staring at me so I could see his hard stare in the rear view mirror and so I would be made anxious as I drove. It was very off-putting.
He did this without there having even been an argument. It was a passive-aggressive thing on his part.
I think he did this to unsettle me when I was in control of the vehicle, another power tactic.