You go back to where you bought the building - so, if it's Robin, go to the carpenter's, choose 'construct farm building' and there will be an icon (below in red) which will take you to your farm screen, where you can select which building to move!
Go to robin's shop. There should be an icon at the bottom that if you navigate to with your controller will say that it is for moving buildings. Click/Press/Tap on it and you should be able to move your building