The world is full of unethical greedy slime-balls who will sue over anything, as evidenced in this forum from time to time. They'll sell their soul to take a chance at hitting that lawsuit lottery.
There is nothing you can do to stop a lawsuit from being filed, but in this case it would be so frivolous all you would have to do to win would be to show up.
PS
STOP TALKING TO YOUR NEIGHBORS! 