I always found Protector of the Valley, the one wherein you had to complete every Adventurer Guild Monster Slaying quest, to be perhaps the most tedious, if not the most difficult. It felt little more than a boring pointless grind for an achievement that was effectively meaningless. I mean, sure, you got some cool toys along the way, but they were generally gated behind extremely grindy quests. Probably the most iconic being the Slime Ring, which requires so many slime kills that I generally just... don't bother. Not because I can't, but because it just isn't worth the effort.
Likewise with A Complete Collection (filling the museum collection). You're always hunting That One Artifact You Can't Find, and due to how RNG works in the game, there's a few that are very annoying to obtain, especially if you have an aversion to fishing as I do. Given sufficient time, it's certainly doable. It's even possible to do relatively quickly (I've seen posts of people achieving it before the end of the first year), but that generally requires substantial luck more than anything else. As long as you can unlock the sewer, the rest are meaningless, outside of this one achievement.
Of course, the most difficult would be Fector's Challenge, hands down. Obviously. And it's one that I simply haven't bothered to try for. I don't need to get all achievements for every game, and that is one that just has too high an entry barrier for me to bother with. I suppose, if it really irked me, I could use a mod to 'cheat' it, but that seems to be unsportsmanlike to me. But it's such a non-issue for me that I can't be arsed to care enough.