Interview- Anil Jain - Image Processing

You know, I’ve been working in the general area of pattern recognition, image processing, computer vision for the last 40 years. And it just sort of happened, the serendipity, that in 1990 somebody called me from Washington, D.C. and said, you know, you do good image processing work, and the NSA has designed or funded the development of FPGA processor. And we can give you this FPGA processing board, which is attached to a Sun, at that time, 350 workstation, and we can give you some money. Can you find some civilian application of this, okay? So we thought about, as you know, FPGAs, field programable gate arrays, can be reconfigured to do both low-level operations and high-level operations. So in thinking in terms of the image processing domain we said, what it is that we can do? So low-level operations is easy, convolution. We look at the pixel, look at its neighboring pixels, and you can do either smoothing or edge detection and that’s some kind of a local filtering operation. For the high level, what could we do? Well, point matching is a generic computer vision operation. If we do stereo correspondence you have left image, right image, and pick some landmarks in the two images and align them and then estimate the depth. So we said, well, what would be an application of this? And this is where we just thought of, well, fingerprint. Because fingerprint matching is essentially a point-matching operation. If you watch any crime show on the TV these days, CSI or anything, they will show you a computer extracting the minutia points from a fingerprint and doing the matching. So it’s really fingerprint matching is essentially a point matching. And then the local operation filtering operation means images, fingerprint images, when they’re captured are generally of low quality, a little bit blurred, and so on. So we need to do some enhancement image enhancement of that. So that’s how we really got started 25 years ago in fingerprint recognition. And I think now we have developed several technologies in fingerprint which has made some impact. And I think one of the best things which we have done is called texture-based fingerprint matching. Traditionally, fingerprint matching is based on points. But what happens if the fingerprint image does not have sufficient number of points, or the image quality is poor, that we can not extract a sufficient number of points? That’s when you need to look at the texture, image texture formed by the ridges and valleys’ which characterize a fingerprint. So one of my PhD students, Salil Prabhakar, around 1999, 2000, came up with a bank of filters which will capture the texture characteristics of the fingerprint and that can be used for fingerprint matching. And surprisingly, not necessarily surprisingly but to our delight, this texture matching is what is used in the small sensors, embedded fingerprint sensors, embedded in mobile phones. In order to have a small footprint of the fingerprint sensor which are embedded in mobile phone, and to reduce the cost, these sensors or fingerprint readers are about 80 pixels by 80 pixels. Traditionally, fingerprint sensors are 512 by 512, images of that size. So an 80 by 80 image captures only a small part of the fingerprint, not the whole fingerprint. And so if you capture only a small part of the fingerprint the number of minutiae points in it is maybe only four or five, and that’s not enough to establish a correspondence between two different fingerprint impressions. So this is where the extra information is used. So I think some of the work we did 25 years ago, 15 years ago or so, in terms of texture matching is now seeing some renewed interest. So, look forward. What’s got you curious now? Well there’s a number of interesting things we are doing. So for example, the traditional model of authentication or security for mobile space or any login that you do is log in once and then use it forever. But that was sort of revolved around the desktop computers. Now if you have a mobile phone, it’s easily accessible by other people. So this notion of authenticate once and use forever is really not appropriate for higher levels of security. So that’s why we have this nuisance built into the system where you unlock the phone and after five minutes of not using it, you have to keep unlocking again and again. And a typical person may unlock a phone 40 or 50 times a day. Now you are spending more time with the phone these days than with any other device. So why doesn’t the device learn who you are based on your behavioral patterns, based on how you swipe the screen, based on how you hold it, GPS. And even the camera can turn on once in a while to capture your partial face image, and it knows, yeah, this is the owner of the phone. So this mode of operation is called continuous authentication. So in the morning, you can present your frontal face image to the camera to get a strong authentication, or you can use a fingerprint, and then the rest of the day, unless the system is really unsure that it’s not the owner, the system can just sort of keep it unlocked for you. As long as it is, it has a sufficient confidence that it’s the owner of the phone. So, that’s one area which is quite attractive. The other area, which is a very interesting. And this applies to any biometric modality, whether it is fingerprint, face, or iris. These are the three most widely used body characteristics which are used for identification. Is what is the persistence of these biometric traits, and what is the uniqueness of the biometric trait. So if I have a ten-digit key or ten-digit PIN, I know how many distinct identities I can have. Ten to the power of ten. But can we say the same thing what can we say about fingerprints? How many unique identities can it discriminate? In principle, every finger has a different pattern on it. So there are approximately 7 billion people living on the earth right now. So assuming each one has 10 fingers, there are 70 billion fingers. So in principle, we should be able to discriminate 70 billion individuals using this. But it doesn’t work quite this way because what is on the finger is maybe quite different from the impression of the finger which you use, of the image which you obtain to do the recognition. And this question of what is the individuality or what is the uniqueness of fingerprint or any other modality is not known. And this is something which we would like to, which we are working on. How do we establish the probability with which we can say that these two fingerprints are distinct? Or the probability that your fingerprint will match one of my fingerprints? That is referred to as probability of false correspondence. And this notion is very important in the legal proceedings, especially when a person is convicted based on partial fingerprint found at the crime scene. And there have been a lot of innocent individuals who have been incarcerated because of the wrong testimony or wrong conclusions. So basically we need to have some scientific basis for understanding what is the probability of false correspondence. The second thing, the second fundamental premise of any biometric trait, is the persistence. That is, does the fingerprint pattern change over time? Does the iris pattern change over time? We know that the people age so the face characteristics changes over time and it is indeed too that state of the art face recognition systems start falling apart when the age separation between two images of the same person exceeds about ten years or so, okay. But in the case of fingerprint and iris, we have been led to believe that they last forever, but there is no scientific proof for that. And so just recently last year, we conducted a study of about 20,000 individuals. And the data came from Michigan State Police, because they are the ones who often encounter the same person again and again. And over a 15-year period, these 20,000 or so individuals had been arrested multiple times. So we have their fingerprint impressions on multiple encounters with the police. And we showed scientifically using a multi-level statistical models, that fingerprint accuracy over this time period does not degrade. So those are the two important questions. The third important question is, if biometrics is going to replace passwords and ID cards, or used in conjunction with it, what would happen if somebody steals your biometric traits? That means that the template or the representation of the fingerprint which is stored in the databases, whether it’s in your mobile phone or whether it’s in your local bank. How do we secure it so that even if it is stolen, it cannot be used or you can reissue a representation. That is, it’s like a type of a credit card number which can be revoked and then issued a new credit card number. That is, I collect your fingerprint, but I never use the original version of it. I use some transformed version of it.