I think it's a great idea. Contact
https://baclocal1.org/ and tell them your plans. If they aren't helpful try something else. Just know that a full day of physical work is a whole different type of being in shape than a gym workout.
I worked as brick laborer as a teen. I was young and dumb and...
anyway, it was strenuous work but I enjoyed it.