[M.D. Holloway]

There are a few things that come to mind:
• What type of a voltmeter is being used? If this is a low amperage circuit and the voltmeter requires a healthy amount of wattage to work, then the overall voltage will go down because the voltmeter is sucking out the power.
• Considered a larger battery? If space is not a problem, then a larger battery may be better able to maintain the desired voltage over time.
• What type of battery is being used? A deep cycle battery will maintain maximum voltage over a longer period of time than a starter battery.
• What is the application?
• Attaching another battery in parallel may be a thought. This is, again, based on the idea that space is not an issue.