DOMS, or delayed onset of muscle soreness, is that pain you feel in your muscles 24-72 hours after a workout. It's thought to be caused by the micro-tears in the muscle which in turn make the muscle bigger and stronger and the soreness is a side effect of the repairing process.
The soreness (DOMS) is not caused by excess Lactic Acid. That's a myth. What does seem to cause this soreness is any exercise/muscle stress that is outside of your normal activity range. Even super fit people can get DOMS if they go beyond how they are used to training. Eventually your body will adapt to the stress you're putting on it which means these exercises will cause less soreness. Eccentric movements seem to cause the most DOMS. That is when the muscle is lengthened under tension (like lowering the weight during a bicep curl)...so, longer, more emphasized eccentric phase may cause more DOMS.
Pain = A great workout! Not really. It is definitely possible to get in a great workout without being unable to sit on the toilet the next day. Don't let DOMS gauge how well your workout went.
Avoiding the soreness isn't always possible, especially when starting a new program. You can lessen the effects some by slowly progressing through your program. Don't go to big and too heavy before your muscles have had a chance to adapt a bit to the work you're putting on them. Always give your muscles time to recover between workouts to avoid doing any serious damage to them. Also, make sure you're doing a dynamic warm up before your workout (think movements to warm the muscles up to what they will be doing next, no holding stretches on cold muscles) and save the static stretching for afterwards where you will hold those stretches for at least 30 seconds to give the muscles time to 'let go' and allow to be stretched.
There are some things you can do that are thought to ease DOMS symptoms. Reviews seemed to be pretty mixed though so you can try them out and see if they work for you or not at your own risk. Things like alternating ice and heat, massage or foam rolling, epsom salt soaks, and anti-inflammatories all have been said to help reduce DOMS.