The focus of this study is on computer automated perception of human emotion. We explore the use of Fisherfaces for the recognition of human emotion in facial images. We train a multitude of Fisherface models and evaluate their performance against an independent test set. We build and test a compound hierarchical system that attempts to interpret human emotion in real time using face detection and tracking algorithms in conjunction with our facial expression analysis methodology. Our results indicate that Fisherfaces can be useful in predicting emotion based on content retrieved from facial images. We note that, with this approach, some emotions are more easily predicted than others. We also suggest that a compound hierarchical model is more effective than a single stand-alone Fisherface model.