This happens to me too and I play on switch! He keeps on getting stuck in the fence while he's on the way to visit his grandparents. Just maybe... is he trying to spy on us...?
I placed some house plants on the path in front of the fence so that he pauses, gets the ? icon above his head, then walks right through them to go straight to town. I'm not sure if it's a complete fix, but it might help!