You need co/decs. Compression/Decompression algorithms.
http://en.wikipedia.org/wiki/Codec
Basically every format has it's own plus you can have an avi that uses different schemes further complicating things.
I am a codec whore, I grab everything I can find.
Start with ffmpeg, it's an open source converter project that will install/get you access to the most common stuff.
Good Luck.