Would it be better if there was no chance of getting a treasure chest while trying to get the Legendary Fish?
I don't think so. I mean, it's possible that catching the legendary fish without being distracted may still end up with you getting the treasure chest, and getting a treasure chest increases the experience you get from the catch when you make the catch. I'd hate for the chance to get bonus fishing experience on legendary fish went away.
As you already pointed out, also: yeah, more than one legendary fish is easier to catch than more than one non-legendary fish. I disagree that there are so many more factors involved, though. All you need to do is be in the right area in the right season, and in one case in the right weather (and thankfully that weather is one you can choose if you want it). They are all available literally all day. I mean, that means that all legendary fish are even easier to fit the requirements for than the pufferfish.
some Legendary Fishes were quite easy to catch. Some were not.
The difficulty on the two easiest (glacierfish and mutant carp) fish are lower than the hardest non-legendary, which can make a difference. What makes a bigger difference, though is the style of movement for the fish. Most legendary fish are "mixed" behavior, which means it can be any behavior and it can change... and the randomness involved could work to your favor by picking a behavior that is really predictable (like smooth or floater), and other times it'll screw you by switching back and forth between two more difficult behaviors (like dart and sinker).
