I am one of those crazy people who loves skull cavern enough to go in massively underpowered just because I enjoy it. That said, it took me a long time to get there. Bring a lot of food, that's essential. Gold cheese isn't the only thing that works for that, but something that gives a decent chunk of hp at once is important because otherwise the hits taken between eating can kill you faster than you can heal.
Rings aren't entirely necessary but are very nice.
Galaxy sword isn't necessary but I'd highly suggest buying the nicest weapon available in the adventurer's guild which I think is the lava katana. A weaker weapon will take so many hits to kill monsters it's very frustrating.
Staircases are also not necessary but will make the loot nicer. Loot gets substantially better the deeper you go, so skipping the first few levels will make the trip more worth it. Plus, it's nice to be able to leave if a level gets too overwhelming.
Food (and/or drink) with a speed buff is very nice. I'd rather have a speed buff than anything else in skull cavern. There are a bunch of other helpful buffs as well - search skull cavern in the forums and you'll get plenty of recommendations.
A fully upgraded pickaxe is, once again, not strictly necessary but extremely helpful. I'd suggest at least gold pickaxe.
Bombs are great if you can afford them, as is explosive ammo for a slingshot. It makes it possible to kill mummies, and makes mining go faster.
Finally, skull cavern at all is not strictly necessary. It's the best source of iridium but not the only source. My first save I went into skull cavern once, nope'd my way right back out, and got all my iridium from the statue Grandpa gives you third year. It's a game, if you're not having fun then don't do it.
Finally - I'm curious about ending up in skull cavern when doing regular mines. I think there is a bug that does this sometimes - can someone else speak to that? Or maybe you're running into the skulls from the quarry mine? Once doing the quarry mine once, they'll occasionally show up in the regular mines. They're super annoying imo, I normally just staircase away to skip that level unless I need a specific thing they drop.