Here's an interesting study that found that autistic subjects experienced greater amygdala (emotions) activation with direct eye gaze than non-autistic subjects. The study also found that autistic subjects were not more likely to look at 'mouths' than 'eyes' as had been found in a different study looking at motion pictures. See a previous post on the Visual Side of Autism here. Direct eye contact can also overwhelm young children's visual working memories. This may happen to some people with autism as well.
Strong Amygdala Activation in Autism