1/54 is still pretty long odds.
As for "the later years" let me tell you about my big (I'm 29 and she's 36, but she's definitely smaller than me!) sister, Blythe.
She has been living independently in her own apartment in a "normal" apartment complex since she was 21.
She plays the violin (can do all of Suzuki book 1) and sings in a chorus.
She works full time for a non-profit doing receptionist work.
She owns a Volvo and has had a drivers license since she was 26... perfect driving record.
She drives 3-4 miles, then takes the subway in to downtown Boston for work every week day.
She does her own shopping, cooking, pays her bills, etc.
She has a computer and uses it for email, checking her bank account, etc.
She has a more active social life than me.
She live about 20 miles from me here in MA, and our parents live in Utah... no problem.
The bottom line is that someone with DS can live a relatively normal, productive life. Granted I've never known an existence without my sister, but from my perspective it doesn't seem like my family lives under a DS cloud or is really any different than any other family. That's not to say the challenges aren't significant... they are. Being born with DS is definitely being born behind the 8 ball; no point in sugar coating it.
This might seem like an odd analogy, but I would compare the impact of having someone in my family with DS in their "later years" to having an elderly relative... you feel the need to check up on them and there are occasional logistical issues, but that person is essentially independent. My parents are comfortable living half way across the country knowing that I live in the vicinity to help with small stuff... for example, I fixed her vacuum cleaner on Sunday.
All that being said, don't start worrying too hard about something with 1 in 54 odds. No matter who your kid ends up being, you're going to be their dad.
Here is a gratuitous pic of me and my sister.