Thursday, August 21, 2014
Researchers from Australia’s University of New South Wales (UNSW) and the United Kingdom’s University of York and University of Aberdeen have measured face matching aptitude of 49 Australian passport-issuing officers and compared it against a control group of 38 students from UNSW. In the test given to both groups, both had a roughly 20% error rate. The study “Passport Officers’ Errors in Face Matching” was published in PLOS ONE this Monday.
Previous psychological research has shown difficulty of deciding if two photographs are of the same stranger or of two different unfamiliar people. As coauthoring psychologist Dr David White says, “Despite this, photo-ID is still widely used in security settings. Whenever we cross a border, apply for a passport or access secure premises, our appearance is checked against a photograph”.
In the person-to-photo test, 30 officers —with mean age 48.0, 21 of them female— in Sydney Passport Office, during a normal working day, had to compare photograph (presented for up to ten seconds) to the person present. 34 students, 17 of each gender, were recruited as ‘applicants’ for the identification task. The person-to-photo test could not be given to a control group because it was not possible to bring back the ‘applicants’ to repeat the test.
ID cards were generated, genuine and ‘fraudulent’, for each ‘applicant’. Photographs of the ‘applicants’ were just a few days old. Each applicant’s ‘fraudulent’ photo was subjectively chosen for greatest similarity to their genuine photo.
The ‘applicant’ did not know whether the ID they presented was genuine or ‘fraudulent’, to avoid giving any cues to the passport officers. The passport offers rejected 6% of the genuine photos and accepted 14% of the ‘fraudulent’ ones, for 10% wrong decisions overall.
As coauthor Rob Jenkins, a psychologist at the University of York, said: “This level of human error in Australian passport office staff really is quite striking, and it would be reasonable to expect a similar level of performance at UK passport control. […] At Heathrow Airport alone, millions of people attempt to enter the UK every year. At this scale, an error rate of 15 per cent would correspond to the admittance of several thousand travellers bearing fake passports”.
One week later, 30 officers in Sydney Passport Office took the short version of Glasgow Face Matching Test (GFMT); 28 were participants from the person-to-photo test, and two were replacements for officers from the earlier test who were absent from work the day of the second test. GFMT performance predicted performance on the person-to-photo test for mismatches, but not for matches. The researchers suggested this could be explained by an upper limit on how accurate person-to-photo match trials can be.
The face matching accuracy was found to be independent of experience and training. As Dr David White says, “passport officers did not perform better, despite their experience and training. They made a large number of errors, just like the untrained university students we tested”.
Two years later, the photo-to-photo test was conducted, with images supplied by 21 ‘applicants’ from the person-to-photo test. 27 passport officers —mean age 45.5, 10 of them participants from the person-to-photo test, 22 of them female— as well as 38 volunteers —mean age 18.9, 26 of them female— compared photos taken recently to photos either from two years earlier or from official identification documents. All variations on the trials taken together produced 84 trials, ordered randomly. The testing was again done at the Sydney Passport Office, without time restriction, to encourage accurate response.
A possible solution of the problem is recruitment of staff with high face-matching accuracy, as Dr David White notes: “But we observed very large individual differences. Some passport officers were 100 per cent accurate. This suggests security could be significantly improved by using aptitude tests to select staff for jobs involving photo-ID checks. Because of this study, the Australian Passport Office now sets face-matching tests when recruiting staff and when selecting facial comparison experts”.
Match accuracy tested lower with official identification photos than with two-year-old photos taken by researchers; average error was 29.1%, with volunteers especially inaccurate on the official identification matches. Mismatch average error was 10.6%. Australian Department of Foreign Affairs and Trade has stated passport photo specifications comply fully with International Civil Aviation Organization (ICAO) standards.
Another solution Dr David White mentioned is using multiple photographs: “One of the more recent papers we’ve released shows that if you have multiple images — not just a single snapshot — there are quite large gains in performance”.
As coauthor Professor Mike Burton of the University of Aberdeen says: “There is a great emphasis on a passport image to fit all purposes but people often comment on the fact that their passport photo looks nothing like them […] It seems strange that we expect a single passport shot to encompass a person and allow us to consistently recognise them. Could there in fact be an argument for our passports to contain a multitude of images, taken at different angles, in different lighting and formats?”
The research is supported by the Australian Passport Office within Department of Foreign Affairs and Trade and also funded by the Australian Research Council.