He was looking at the attractive woman just past you at the next table.
Or he may have been looking at the chalkboard with the list of desserts that was just above your head, trying to decide what he wanted.
Just kidding... I stole that last one from a movie or TV show.
