Sorry you're having issues connecting! I'd say the first thing to make sure is that there is no issue with you or your friend's internet. This sounds like it is likely a problem with the internet connection itself, rather than a bug in Stardew Valley. I'd keep trying it a few times, maybe have both you and your friend reset modems, etc. If none of that works and the problem keeps happening ( and especially if other games work between you & your friend, but only Stardew Valley fails), let me know.