You have to go through and carefully clean everything out in terms of any mod-added items, spouses, etc. And then—this is very important—you must save a day with SMAPI and Error Handler installed, which helps to clean out all mod content (there will be warnings that day, but they will go away after that). Once your game plays fine (no errors in log, no warnings) with just SMAPI + Error Handler, you should be able to play as normal in the vanilla game.