I agree with PoppyForPenny, the Spouse Rooms matter quite a bit. You can see them all here, other than Krobus, and can see Krobus' on his page.
In singleplayer mode, Marriage is possible only with a villager who is marked as "single" in the Social tab.
stardewvalleywiki.com
Krobus is the only friendly monster players will encounter, however he still refers to other hostile monsters as his friends. He is a shadow person who lives in the Sewers. He sells a variety of rare goods. He is also available as a roommate.
stardewvalleywiki.com
Some notes: Shane's is just an outright worse (significantly worse!) version of Abigail's. It's hers if she was a slob, had no musical talent or interests, no hamster, and no sense of adventure. Alex is the only one with Green Flooring, Krobus is uniquely Blue/Green in both the walls and floors, and Leah's is very red. Beyond that, they're all mostly brown, if you're trying to color coordinate. For Penny vs Emily specifically, you're choosing between a pinkish, kinda empty room with a parrot and palm tree for Emily, vs a blueish room full of bookshelves for Penny. If you really like books, parrots, or palm trees, maybe that contributes to the choice.
It can also make sense to base it on what best fits thematically with your character. Penny is bookish, and all about kids, learning, and community. Emily is free spirited, and is all about crystals, dancing, and making clothing. If either of those themes appeal more to you as a person or make more sense for your character, then go for it! As others have said there's no "wrong" choice, and you Can get divorced if you have regret later. However doing so has (semi) permanent impact on how those you've divorced interact with you, so I wouldn't go around just marrying and divorcing everyone, personally.