The face conveys a wide range of cues. Facial expression can signal how an individual is feeling, and gaze can signal where their attention is focused. Integrating information about others’ expressions and gaze influences how we direct our own attention, form impressions of others, and assess threats. This study was designed to explore how people with certain personality traits such as alexithymia and sensory processing sensitivity (SPS), as well as those with mental health problems characterized by anxiety integrate expression and gaze cues when viewing faces signaling two different kinds of “threat” cues: anger and fear. Participants N = 74 (49 female; Mage = 20.45 years, SD = 4.64) with varying degrees of alexithymia and SPS were administered self-report questionnaires on their mood, and levels of anxiety, social anxiety, and anxiety sensitivity. They then completed two tasks requiring speeded judgments in which they decided if an expressive stimulus face was looking at them or not (gaze task) and if the stimulus face was angry or fearful (expression task). A series of ANOVAs and exploratory correlation analyses were used to test the relationship between variables with task performance. Results revealed that: (1) the order in which gaze and expression judgment tasks are completed is an important determinant of task performance; (2) the cone of gaze is more symmetrical when viewing angry (vs. fearful) faces; and (3) anxiety-related problems, SPS, and difficulties with emotional appraisal are negatively associated with problems in fear processing when gaze is strongly averted and viewers are focused on performing mental state attributions and self-directed threat assessment. This research furthers our understanding of individual difference factors that may influence the form of gaze-by-expression interactions that help shape one’s social perception. Additionally, the findings emphasize the importance of investigating the differences between left and right averted gaze.