So the problem did turn out to be the switch.
The new switch has a slight fitment issue and you need to order the nut for it and just tighten it down real good. You also have to get out the bentley and an ohm meter and figure out which pins of the new switch get what wires (it is different from original switch). But when all was said and done that was the issue, have good working fan in all positions now.
