You have to be allowing uploads - make sure that ports 6880-6889 are forwarded from your outside connection to your computer (assuming you have a router and are doing NAT)
When no-one else is seeding the file, then the primary seed isn't getting hammered and you can still download directly from it at full speed.
Edit - when I BT a new linux distro, I typically allow uploads but cap the rate at 5k/sec.