You're right in that I'm fertilising Melons, because my remixed Quality Crops Bundle requires 3 out of either gold-star Cauliflower, Melons, Blueberries, or Pumpkins.
I don't particularly like planting lots of Cauliflower, so that was off the chart, but now that I'm farming level 7 (just started Summer), the 9 Melon Seeds that Gunther gives + the 30 that I bought should be enough, Blueberries are easy peasy, and Pumpkins follow the same reckoning as Melons.
And had I remembered, I'd have put fertiliser on my 90 Hops and 54 Blueberries, but I forgot until it was 1:00am.
But in Spring? I was fertilising basically everything. It's totally worth it, I ended up with more Gold and Silver crops than not.
It also provides a nice division line between what I should process/keep (normal), what I should sell (silver), and what I should keep for gifting (gold).
But I never thought about NOT fertilising crops. Thanks for providing an alternate perspective!