Are you talking about a E46 vs. (E39 or E90?)
The E46 is a bit too heavy at the autocross but great on the street. You will notice the extra weight of the E39 even on the street. I think the E90 is lighter than the E39 and the E90 is only about 200 lbs heavier than the E46, each with the M52 3.0 engine.
Since you are coming from a Porsche I would have to suggest the 330 Sport or PP.
Buy mine
Just reread you post and you are talking about an E39, sorry. The E39 is bigger and heavier and you will notice it, but hey it hauls 5 comfortably and is great fun to drive.