Glaring Errors in UIDAIs Rebuttal

The article was published in Economic & Political Weekly Vol. 51, Issue No. 36, September 3, 2016.

While I am not a statistician, I have followed the technical debate between Hans Verghese Mathews and the UIDAI closely, and see a number of glaring errors in the latter’s so-called rebuttal in EPW (March 12, 2016).

The UIDAI alleges Mathews to have ignored the evidence that the Receiver Operating Characteristic (ROC) “flattens” with more factors. However, Mathews cannot be accused of ignorance if the flattening of the ROC is not relevant to his argument. To explain this in simple terms, the ROC curve is used to choose the appropriate “threshold distance” which determines false positives and false negatives, and belongs to a stage which precedes the estimation of the false positive identification rates (FPIR).

However, Mathews has used the FPIR estimates provided by the UIDAI (based on evidence from the enrolment of 84 million persons), and calculated how the FPIR changes when extrapolated for a population of 1.2 billion persons. In other words, he did not need to look at the ROC curve as that factor is not relevant to his argument, since he has used UIDAI data (which has presumably been estimated on the basis of all 12 factors : 10 fingerprints and 2 irises).

Further, UIDAI asks why Mathews has assumed a linear curve for his extrapolation. Mathews has done no such thing. In fact, in their paper “Role of Biometric Technology in Aadhaar Enrollment,” the UIDAI states: “FPIR rate grows linearly with the database size” (nd, 19). Thus, this is an assumption formerly made by them (without providing rationale for it to be a linear curve as opposed to anything else).  Mathews mathematically derives bounds for the FPIR in his paper, that is, the range within which the FPIR lies. One gets a linear curve only if they use the upper bound and not on the usage of anything else. So while Mathews does, as he explains, provide the results of the calculation based on the upper bound for the sake of simplicity, he nowhere asserts nor assumes a linear curve.If, as the UIDAI claims, one cannot perform such an extrapolation and needs to depend on “empirical evidence” instead, the question arises as to how the UIDAI decided to scale up the programme to 1.3 billion people given the error rates. One could also ask if the machines being used to capture biometrics are good enough for the enlargement. Surely they would have performed some extrapolations to decide this.

In their paper they note that “although it [FPIR] is expected to grow as the database size increases, it is not expected to exceed manageable values even at full enrolment of 120 crores” (UIDAI nd, 13). They do not illustrate the extent to which the FPIR is expected to grow—neither in their initial paper, nor in their rebuttal to Mathews—whereas Mathews provides a method of estimating the increase of FPIR. Even if UIDAI is correct in its appraisal of FPIR and that it will not exceed “manageable values,” they need to either exemplify their calculations or release the latest data. They have done neither, and that is quite unfortunate.

References