N Dastigiri Reddy can never be sure if the candidates scoring top marks in the entrance tests to the MSc chemistry programme he teaches at Pondicherry University are the best ones to admit. When asked why atoms of zirconium and hafnium are the same size, a candidate may pick “lanthanide contraction” from the list of four answers given. “But how do I know the fellow understands what that is and has not simply memorised the answer?” asked Reddy.
He has found on several occasions that those who joined the programme with top ranks actually knew and understood very little of the discipline. His colleague in the Centre for South Asian Studies, Subramaniam Raju, had much the same experience. “Students’ rank in the entrance test and analytical skills displayed in class often do not match,” Raju said. “They do not always perform in class – do not ask questions, participate in discussions and debates.”
Pondicherry University uses multiple choice questions for admission tests into postgraduate courses across disciplines – candidates are not required to write an answer, but pick the one they think is correct from four options already given. These are used in large-scale entrance tests for engineering and medical colleges. Many teachers question the suitability of multiple choice questions for admissions at the postgraduate level, particularly for the social sciences and humanities. The latest university to join the debate is Jawaharlal Nehru University, which has announced plans to switch to online entrance tests based on multiple choice questions in 2019.
So far, at least for the social sciences and humanities, the university has conducted pen-and-paper tests that required candidates to write long, essay-type answers allowing the teachers to gauge their creativity, writing and analytical skills.
But a 12-member committee established to study whether Jawaharlal Nehru University can make the switch, cited the example of Pondicherry University in its report and suggested that “the examination pattern be changed permanently to an analysis of domain knowledge that will be based on MCQs [multiple choice questions] with no written skills or subjective questions involved”. This has been interpreted, by teachers and students, as a memory test that will encourage cramming.
In Delhi University, which adopted multiple choice questions for entrance tests into all postgraduate programmes in 2016, many teachers are unhappy with the change.
The case for multiple choice questions
Despite his reservations, Raju does see merit in multiple choice questions for two reasons – they guarantee objectivity in marking, he feels, and when the number of candidates is large, relieves teachers of the responsibility of checking. They have eliminated “the space for bias”, he said. “It is the only way we will be safe from complaints and criticism.”
These twin advantages were mentioned by the JNU panel as well, which noted that the university conducts entrance tests for its 169 programmes (majority of them postgraduate) in 72 centres in 54 cities.
So far, holding exams in these 54 cities has meant sending staff to all of them and faculty-members having to correct papers. “The responsibility of maintaining the sanctity of the evaluation process in terms of preservation of the answer sheets that undergo a subjective evaluation is also a challenge,” says the report. It argued that the reform and outsourcing the conduct of the test to another party, will “minimise man hour losses to the university” and be eco-friendly. This decision is yet to go through the universities top statutory bodies – the academic and executive councils.
For Kavita Singh, whose School of Arts and Aesthetics in Jawaharlal Nehru University, receives about 300-400 answer scripts, evaluation is not a problem. She said the non-teaching staff were delighted with the opportunity to travel. She finds the “urge to standardise a little odd”. However, she added, departments such as history and political science that get thousands of answer scripts may have felt the pressure more.
The situation is different elsewhere, and varies with changing admission policies. Till a few years ago, around 30 teachers in Delhi University’s history department corrected 70-100 answer scripts each, which was manageable, said Upinder Singh from the department. But in 2016, once the university started holding entrance tests in five cities in addition to Delhi, the number of candidates rose to about 5,000, and correcting answer sheets became more difficult. In 2017, the varsity added another five cities where entrance tests are held.
The response to online tests has been mixed. Delhi University has faced teething problems this year – admit cards were issued just two days before the exam held on Sunday, and candidates discovered they had been assigned examination centres in parts of Delhi that were difficult to access. But at least Delhi University sets its own questions. At Pondicherry University, heads of department suggest names of experts, the private company conducting the exams gets them to frame questions and, on occasion, university faculty members rue the standard of questions set, as Reddy did for chemistry in May. He had reported his views on the paper set but it was too late to change it.
The problem with ‘domain knowledge’
While JNU’s student activists have resisted the shift to an online test, arguing it would be “exclusionary for students coming from rural areas”, teachers are more concerned about the content of the tests.
The Jawaharlal Nehru University panel felt the multiple-choice format “brings objectivity in evaluation and removes subjectivity in analysis” and yields an “accurate and efficient system for evaluating a large number of students”. But teachers, especially those from the social sciences, are not so sure.
“The idea of having a question to which there is only one correct answer is problematic,” said Upinder Singh. “We worried that what we will be testing will be memorisation of facts. Analytical skills, comprehension skills come out better through longer, discursive answers.” The problem only increases at the postgraduate level.
The emphasis on “domain knowledge” implies that a domain – or a syllabus – must be clearly defined. This poses a unique challenge for disciplines that have few undergraduate programmes in the country and therefore no standard undergraduate syllabus the test can be based on.
“I cannot fathom what to ask,” said Kavita Singh. The School of Arts and Aesthetics offers MA programmes in cinema studies, art history and theatre and performance. There are very few undergraduate programmes in the country teaching elements relevant to any of these three specialisations.
“We get all kinds of people – from sociology, medical doctors, pharmacology, engineers – and the [essay-type] exam had the capacity to test their skill and aptitude for the subject without assuming they would have gone through a standardised syllabus on the subject at the undergraduate level,” she said. “Open-ended questions” assessing skill and aptitude rather than memory allowed the humanities and social sciences to draw candidates from other disciplines. Plus, leaving scope for subjectivity is not a disadvantage.
For example, this year, the department had asked applicants to compare the two versions of Blade Runner – 1982 and 2017. Applicants had to pick one and defend their choice. As Kavita Singh explained, the School of Arts and Aesthetics would not have selected students on the basis of which version of Blade Runner they liked. “My views about any particular version of Blade Runner are not important,” she said. “What matters is why [the candidate] picked one and how well they were able to persuade me. Have they been able to defend their opinion? We are interested in their subjectivity.” The problem with the multiple-choice questions, she said, is that it has “no room for knowledge that is not right or wrong”.
Upinder Singh, from Delhi University, agreed that the “current system discriminates between students who want to switch streams from bachelors to masters”.
Before the change to multiple-choice questions, the entrance test paper for the MA History course in Delhi University was divided into two sections, one designed specifically for students who had not studied history in their undergraduate studies, she said. “In that we asked different kind of questions because we were aware there were students who would do very well in a MA programme but were from a science background or an allied discipline like political science, sociology or literature,” she said. Multiple-choice questions, on the other hand, require paper-setters to follow the syllabus – in this case, Delhi University’s BA (Honours) History – more closely.
At Pondicherry University, the Centre for South Asian Studies draws questions from a host of allied disciplines taught at the undergraduate level, including international relations, political science and research methodology. “But in the social sciences, there can be different interpretations and debates that require originality, creativity,” said Raju. “That does not come through in the multiple-choice questions.”
Objective questions and the sciences
The common assumption is that multiple-choice questions work for the sciences and mathematics. Reddy considered it the best and most objective format possible till Pondicherry University taught him otherwise. For one, he thinks the format has made it easier for private coaching centres. “They make question banks and have students memorise answers,” he said. “Questions in these tests are often repeated too. This would not have been possible if candidates were required to write a few lines as well. A student can memorise which way a chemical reaction goes but still not understand or be able to explain why it cannot go differently.”
Worse, habits of cramming are carried to class once candidates take admission. “I have had students write just the term without explaining it and argue with me over marks,” he said, citing the same example of lanthanide contraction. The question had been allotted four marks and demanded a “comprehensive answer”.
Looking for alternatives
The departments at Delhi University at least had suggested other formats in 2016. “Our department had suggested analytical questions based on passages and shorter answers or a combination of these with a longer answer,” said Upinder Singh, from the history department. “Some of these alternatives should have been thought through and there could have been some experimentation.” There was none. Nor have the universities attempted to assess the impact of the change on the quality of students they were admitting
With essay-type answers and marking spread across many teachers, “standardisation is an issue”, she conceded but added that “it is possible to work around with some clear guidelines”.
Reddy believes tests would not reward rote-learning if there were a few short answer questions and the multiple-choice ones were more carefully designed. “It is possible to set good multiple-choice questions if the paper-setters spend a bit more time making them more confusing or indirect,” he said.