In Video Forensics we are dealing with the video quality issues more than the investigation objectives on hand. This can be even harder with subsequent degradation of the information quality contained in the video which is crucial to investigation. Facial identification for instance, is made harder by the recording factors e.g. noise, lens distortion, resolution or compression artifacts which are always found coupled with other factors for example facial pose, facial orientation, illumination invariance and occlusions. Therefore, biometrics analysis in video forensics is a tough nut to crack.
Biometrics technology, by traditional concept is a system which always requires cooperation from users at any degrees and form. Fingerprint for example, requires a user to cooperate by giving the machine his or her finger. Other example such as iris recognition system, requires the user to provide their eyes in order to verify who they are in order to get access to certain room or facilities. In forensics however, there is none. The biometrics evidence is as the way as it was being recorded by a device (for this case its the CCTV)… and of course no criminal would be smart enough to smile at the cameras for that matter.
I was a firm believer of traditional biometrics – to uniquely identify an individual by the characteristics they owned. Years of involvement in this research has really build me up thick with such idea. When I joined forensics and was asked to develop a methodology and SOP for face recognition, i was shocked by the nature of evidences shown up in our investigation. And for the first time, I questioned on my every stand and understanding of the technology. I mean even if its to identify a face in a video exhibit it would take a lot of effort and resources to ensure the result is accurate and robust. In my opinion, with no whatsoever cooperation from the user, it is hardly called a biometric system. For example, how many of us really care to face the surveillance cameras when we see one? Yes, I can guarantee the odd is almost to none. By the thumb rule of face recognition, the face should be at full-frontal and at a certain good resolution in the field of view. In video evidence, this is hardly to come by. Most people in the video exhibit appears to be looking away due to 1) the camera position and 2) the facial position in the view. These really affect the forensics analysis.
Therefore, what is really important to an investigation? Is it to uniquely identify the suspect in a video or to uniquely described the suspect? This is what define soft-biometrics, a method to describes biometrics in a video or image with no engagement of recipient’s cooperation. This is somehow fit to the forensics analysis methodology. According to Simon  :
” Soft biometrics are characteristics that can be used to describe, but not uniquely identify an individual. These include traits such as height, weight, gender, hair, skin and clothing colour. Unlike traditional biometrics (i.e. face, voice) which require cooperation from the subject, soft biometrics can be acquired by surveillance cameras at range without any user cooperation. Whilst these traits cannot provide robust authentication, they can be used to provide coarse authentication or identification at long range, locate a subject who has been previously seen or who matches a description, as well as aid in object tracking. “
So how soft-biometrics analysis is conducted? What features are taken into consideration? According to Simon, the analysis can be conducted by extracting the soft-biometrics models (the head, the torso and the legs) from the surveillance video. The model is then segmented into each section and is treated separately. The crucial part is the segmentation. For that purpose color segmentation is applied. These segmented data can then be analyzed for the facial information, the attires and the Gait information. The setbacks for this methodology is the illumination factors. Therefore, a more robust detection algorithm for example Graph-Cut Segmentation and Active Appearance Model (AAM) should be put into consideration.
The challenges for soft-biometrics to be used in forensics (as I can think of) are all lies in other exhibits seized which can be associated with the probe individuals in the video. For example in confirming the suspect is wearing the same shirt and skirt on the event of the crime, the law enforcement should also seized the attires that is in the belonging to the suspect. Furthermore, to incorporate facial and gait analysis, the suspects facial and the way he/she walks should also be recorded as enrollment. In other word, the law enforcement should think of a way to get these information in conducting forensics analysis in their investigation.
There is a case where we applied a similar methodology. For that case, apart from conducting face recognition we were also required to conduct attires matching. The problem was the color of the shirts they wore in the video and the ones seized from their belonging are the same type but not the color. This is due to the recording setting of the CCTV system in the premise which degrades (due to some unknown reason) the video color to another color space. A stripe of black and red was found to appear blue and purple in the video. To made things worse, to correct the color of the exhibit video is considered bias. Therefore, what we did- we brought all the clothes back to the crime scene and recorded them via the same cameras which recorded the video exhibits. The idea is to establish a connection in term of color obetween the suspect seized clothes to the one they wore on the time of the crime. From there we managed to successfully proof the same attires were being used. Plus, the face recognition results also showed a positive matching of their face to the face of the probes in the video.
As Simon claimed, soft-biometrics is not as accurate as traditional biometrics. Perhaps multi-modal biometrics approaches can further enhanced the current method. If we find both the face and the gait, why not establish the combination of both. Plus with the information of the clothes, or any other unique characteristics of the suspect taken into consideration for describing, the analysis can be without doubt a strong one.
. Simon Denman, Clinton Fookes, Alina Bialkowski, Sridha Sridharan: Soft-Biometrics: Unconstrained Authentication in a Surveillance Environment. DICTA 2009: 196-203