[Zeke]

Don't wander off base. A repurposed neutral, generally the hot side of a switch leg should be taped black or at least marked black. Sometimes it's not done, but anything I can think off does not result in a dead short that is described here.

That having been said, the wiring sounds messed up and way over the OP's ability. The condition I'm suspecting here can be pretty damn dangerous and needs professional attention. Going about electrical work a bit here and a bit there is how wiring becomes a nightmare. People get tunnel vision and don't look at the system as a whole resulting in unbalanced panel power distribution, back feeds and plain old open circuits that are difficult to trace and repair.

And then you have what potentially seems like both phases in the same box. This is not impossible as the box may serve as a splice point for another circuit that has nothing to do with any device present at the box. I would look at box fill as well.