Apparently it's possible to plant a seed on the farm cave entrance, but it's impossible to remove it later on, because either character just walks into the cave or using tools on that spot does absolutely nothing.
I had a tree seed randomly spawn here! Never tried to remove it as a seed but once I could no longer walk over it (it sprouted) I chopped it with an axe just fine.
