After four years of online testing, the Common Law Admission Test is again going offline. A new report explains why: the 2018 test, conducted on May 13, did not go well.
A government committee, set up in September on the Supreme Court’s direction, has found technical problems on a startling scale in this year’s exam: over a third of the candidates had to log in more than once because their screens froze mid-test, or the questions did not appear at all. Of the total 54,465 candidates across 258 test centres, 19,983 had to log in at least twice and 612 at least five times. Fourteen had to log in as many as 10 times or more, while one exceptionally unfortunate candidate had to do so 19 times. This after paying Rs 4,000 for the test. Considering the fee “way above” what is reasonable, the committee also asked for it be reduced to Rs 1,500.
The test is conducted for admission to India’s 19 national law universities and is organised by a different university every year. The organising university outsources the technical and logistical parts of the process to a private testing company. This year, it was the turn of Kochi’s National University of Advanced Legal Studies to administer the test, with Sify Technologies Limited as the technical partner.
Glitches were reported immediately after the exam. According to Bar and Bench, many candidates had lost as much as 10 minutes because of them. Petitions were filed in several High Courts as well as the Supreme Court, which was already hearing a challenge to the test, filed by the legal scholar Shamnad Basheer in 2015.
The candidates demanded a stay on the results and a retest. The apex court, however, allowed the results to be declared by the end of May, the Kochi university established a “grievance redressal committee” to address complaints and, in June, revised marks of 4,801 candidates.
On September 27, the human resource development ministry appointed a committee to look into the complaints as directed by the Supreme Court. It comprised Manindra Agrawal of the Indian Institute of Technology, Kanpur; Vineet Joshi, director general of the National Testing Agency; Ashok Kumar Jaryal of All India Institute of Medical Sciences, Delhi; and Neeraj Dwivedi of the Indian Institute of Management, Lucknow. The panel submitted its report to the Supreme Court on Tuesday and shared with the petitioners, including Basheer, on Wednesday.
‘Not aware of the magnitude of problem’
The report notes that the Kochi university was not alive to the “scale of malfunctioning” until the grievance redress committee was constituted.
The panel had asked for audits or reports of the measures undertaken by both the university and Sify before the exam but did not receive any from either. While the university’s registrar and Sify shared the procedures laid down for central observers before and during the exam, the report says, the panel could “not determine” if they were actually followed since no reports of “pre-examination activities” were give to it.
Sify did furnish “exam controller day end reports”, but “none of the 153 reports mention any malfunction of computer system or examination”. To the panel, this “lack of any documentation” of glitches “came as a surprise”. Its report, therefore, accuses the university of having poor oversight over the exam. It states:
“It appears that the role played by [the university] was primarily administrative and that complete technical control of...all the phases of the examination, including post-exam analysis, was left to Sify Technologies Limited. Except for the preparing and encryption of the question paper, the whole operation of CLAT 2018 (from application forms, preparation of results and counselling) was managed as a turn-key project by the private partner…The standard operating procedure in case of technical glitches was not in place, leading to ad-hoc decision-making in real-time.”
Candidates affected by the glitches were eligible for extra time. To get it, they had to log off and in again. But the report says a large number of candidates may not have been told about this. From data shared by Sify, the panel found that 9,897 candidates across 61 centres had their exam time extended by one to 40 minutes. Of them, 5,974 did not use the extension at all while some of the rest utilised its partially. The reason for this variation in behaviour was a mystery to the committee.
At least 12,930 candidates, the report states, “show no activity” between their first and second logins, “indicating that the scale of failure to start the examination was...rather larger than the number of complaints made by the candidates to the Grievance Redressal Committee”.
Apparently, the law school in charge was in the dark about the problems on the day of the test. “It was surprising to note [the university] was not aware of the magnitude, exact nature of the problems, solutions attempted by Sify Technologies Limited and decision process for determining of amount of time extension neither on the day of examination nor till the constitution of Grievance Redressal Committee,” the report states.
For this performance, the Kochi university spent Rs 2.6 crore. It had, however, made Rs 27.5 crore in exam fees – a profit of over 90%. Although it is required to share 50% of the fees collected with other national law universities, the committee felt this level of profit is “egregious”.
‘No sincere attempt to find out the cause’
Every year so far, the law school responsible for administering the test picked a new technical partner through a fresh tendering process. Because new teams handled the exam every year, there was no “incremental learning” from previous experience and, therefore, no improvement. The committee’s report suggests such contracts should be given for at least two years, with the possibility of a year’s extension.
It also suggests that “government organisations that are dedicated [to] conducting standardised examinations on regular basis” may be “better placed” at handling the Common Law Admission Test.
For the 2018 test, the National University of Advanced Legal Studies appears to have made it easier for smaller companies with less experience in such large-scale testing to bid. “The amount of average turnover of the company and from exam-related activity was decreased in CLAT-2018 tender as compared to CLAT-2017,” says the report. “Tender document is silent regarding the hardware specification for the computers to be used during the examination even though a separate document...on the same has been submitted to the committee.”
When asked to explain the glitches to the committee, representatives of Sify resorted to guesswork. They said it was due to the “high traffic of the data between local area network server and candidate computers”, the report says, but admitted this conclusion was “primarily a hunch”. “It was apparent that Sify Technologies Limited does not know the cause of the technical glitches,” the report says. “It also seems that no sincere attempt has been made to find out the cause of the problem.”
The Kochi university has withheld the Rs 1.14 crore fee owed to Sify for conducting the test. As per the tender agreement, Sify is also required to underwrite the Rs 1 crore legal costs incurred by the university and pay 10% of the fee as penalty. “Service provider may also be barred from applying for future tenders of [the Common Law Admission Test] for a period of at least three years,” the report recommends.