Studies have shown that individuals with Autism Spectrum Disorder (ASD) tend to gaze aversion during social interaction. It also has been observed that autistic people have significant problems in performing social tasks, including face recognition. Researches emphasize the role of facial gaze, especially visual communication in social interaction and learning. In this paper, we propose an Artificial Neural Network (ANN) to model ASD’s deficiency in face recognition. We used Olivetti Research Laboratory (ORL) face database and chose pictures that fitted our desires. The ANN was trained and tested in three trial experiments; in experiment 1 (exp. 1), we used pictures with up-masked faces (the upper half of the faces had been blurred) in order to model ASD’s face recognition problem, in experiment 2 (exp. 2), pictures with normal pictures was used for simulation normal individuals’ face recognition; and in experiment 3 (exp. 3), we used pictures with down-masked faces as a test group. Testing results show a 20.00% error in the exp. 1, 4.44% error in the exp. 2, and 10.00% error in the exp. 3. Based on these results, the proposed network emphasizes the face recognition problem in ASD as a result of eye contact aversion.