Okay, you can forge up to 3 gems onto a weapon, and each does different things. Rubies increase damage, topaz adds defense, jade increases critical damage, aquamarine increases critical chance, emerald adds speed and amethyst adds knockback. You can do 3 of 1 gem, 1 each of 3 different gems, or any other combination you can think of.
Diamonds are a special case. When you use diamonds, it counts as up to 3 of the other gems... at random! So you don't know what you're going to get when you use a diamond, but it's cheaper than putting specific gems on for specific forgings. Now that you've done this, there's no more adding gems... unless you unforge it, removing and losing all the forging done already so that you can start over.
For my galaxy sword I always do 3 rubies.
Prismatic shards are used to add an enchantment. You can add one to your weapon, fishing rod, axe, pickaxe, hoe and watering can. When you put one of these and a prismatic shard in the forge, what comes out will have a random enchantment that can be applied. If you don't like what you got, you can do it again and get a new enchantment. If you re-enchant something, it will lose the old enchantment and get a new random enchantment. When re-enchanting something, the new enchantment will never be the same as the last two enchantments, so you're guaranteed to get something new. Each enchantment or re-enchantment costs the same, 20 cinder shards and a prismatic shard.