Census 2011: Technology

Do you hold an identity in a country which houses more than a 100 crores of people? Each one of us will as the work on Census 2011 unfolds. As Home Minister P. Chidambaram has pointed out that it is the biggest exercise since mankind came into existence and it is nowhere in the world that a government tries to count and issue identity cards to more than a billion people. The dimensions of this entire charade are hard to envisage. It is being labeled as the world's largest exercise in terms of the data captured, manpower involved and the expenses involved but it is also a technological marvel to achieve such goals!It is a matter of great pride that India has till date been able to hold censuses without any breaks since 1872 every 10 years. Struggle for Independence, Partition, wars, floods and earthquakes, have had no impact and have not been able to interrupt the exercise so far.The Head count operations were halted even in England during World War II but it was successfully conducted in India. Why is the 2011 Census being hyped beyond proportions? Why is it so unique and different from its predecessors? It is the setting up of the National Population Register (NPR) which is the database of every single citizen of India. How's that different from what the Census was already doing? The Census is a purely statistical exercise which involves only the head count whereas the NPR involves individual data collection such as your name, relationship to the head of the family, father's name, mother's name, sex, marital status, date of birth, nationality and the details of the present address (residential) etc.After the forms which have been designed by the National Institute of Design in Ahmedabad are filled, they are scanned and then the images are processed using ICR technology. No other country in the world uses such sophisticated technology to process its Census data.This is indeed a technological marvel for the census in India. It has not been achieved overnight and its evolution has been a gradual process which began about fifty years back. It would be interesting to trace this Technological Evolution. Let us see how it all began a few years back.
Census India: Technology used in 1961 CensusPrior to the 1961 census the data entry and data collection was done manually. The 1961 census initiated and adopted the use of 'Unit Record' systems. The mode of data entry were d punching machines which were used for the first time where the cards were inserted one by one. The punch cards consisted of 80 columns which converted the data into machine readable formats. The data processing was done only on 5% of the data picked out from the large data base. Around 70 officers were involved in the data entry, programming and machine operations. Reproducer was used to duplicate the entire set of punched cards.Verifiers and sorter machines were used for data processing. SRTT Printers called Serial Rolling Total Tabulator cum Printer was used for the tabulation and printing of the collected data. There was no provision for keeping any back up for the huge data.
Census India: Technology used in 1971 CensusThe census in 1971 saw the advent of both mechanical and electrical key punching machines for entering data. As against inserting the cards one by one, punch cards consisting of 80 columns were used in a stalk by the machine. Processing was could be done on 15% of the data picked out from the main data collected. Around 90 officers were involved in the data entry, programming & machine operations. An IBM 1401 computer, Card Reader and a Printer were used for data processing. For the first time, provision was made for back up storage of data and data processing was done using large size spools of magnetic tape.
Census India: Technology used in 1981 CensusData entry was introduced for the first time in the 1981 census and 15 data centers were created across the country along with one main center. More than one state was attached to each center.New technological devices which were used to convert paper based information into machine readable form were 'key to disk' systems supplied by ECIL, ICT and GCS. HP 1000, CD-Cyber 730 and NEC-1000 systems were used for the first time to process data.at the National Informatics Center (NIC), New Delhi and the Regional Computer Center (RCC) at Chandigarh. The data processing facilities were not available at the main center. 25% of the data collected could be captured and processed .Around 1200 officers besides the officers at the 15 data centers were engaged for the entire census data processing activities. The requisite software for data validation, editing, processing and tabulation was developed by the officers at the Data Processing Division at the Office of the Registrar General (ORG), India.
Census India: Technology used in 1991The 1991 census saw dramatic and revolutionary changes in the data processing technology. It was during this time that the ORGI launched its indigenous computing facility and installed the Medha- 930 main frame system using the operating system Unix which was connected to the servers at 15 data centers where the data was entered. Transfer of data between the centers was done using magnetic tapes. Four Regional centers at Delhi, Bhopal, Bhubaneshwar and Chennai were created for editing and generating lower level tabulations and reports whereas the main data processing and generating tables at different levels was done at the Data Processing Division of ORGI and the necessary software for data validation and tabulation was developed by its officers. It was for the first time in 1991 census that camera ready copies of the tabulations were prepared in Hindi as well as English for publication. It was during this census that 45% data was captured and processed. This exercise involved around 1200 officers over and above the ones engaged at the 15 data centers. The software was developed indigenously by DP Division officers of ORGI.
Census India: Technology used in 2001 CensusThe 2001 Census was marked by a lot of hardware up gradations at the 15 data processing centers. This was done to incorporate some contemporary the latest technologies like the Automatic Form Processing Technology which was applied using Intelligent Character Recognition technology (ICR). ICR technology turned out to be the most compatible considering the magnitude of the entire data processing exercise and initiated a revolution of sorts for the census activities.The magnanimity of the data processing can be gauged by the fact that 45 NT Servers, 1060 PIII PCs, 25 High speed heavy duty duplex scanners (Kodak) were set up at all the data processing centers. Back up devices like ZIP SLR and DLT Drives were also installed for the first time. This activity involved around 1200 officers and about 500 contract based operators for capturing and processing data.) . Scanning and data file creations went on for 24 hours a day without a break and for the first time, System Integrators were nominated for these operations. Due to the latest technology hundred percent data could be captured and processed for the first time in any census which amounted to more than a billion records. The scanned images were stored permanently in an archive. The need for setting up of regional centers for tabulations was eliminated due to the use of ICR technology after scanning the schedules.This brought down the expenses drastically and saved some finances for the government. The use of this technology not only eased the process but helped the officers in imbibing new skills as the software was developed by the officers themselves and turned out to be an innovative experience for them.
Census India: Technology used in 2011 CensusIn the current 2011 census, the forms have been printed in 16 languages this time which only reaffirms the fact it is indeed the largest such exercise in the world. The ICR Technology that was pioneered by India during the 2001 census has become a role model for major countries across the globe. The improved version of this software is being utilized again for the scanning of the Census Forms at high speed and automatic interpretation of the data in the 2011 census as India already has the expertise to deal with it and it has been successfully implemented in the past.It is evident that the percentage of data captured grew from a mere 5% to 100% over the last fifty years due to the innovations in technology.In 2011 census the scanners that are being used have additional features like image enhancement, removing noises and detection and auto correction of images through its own software.The version of ICR software that is being used this time has better recognition featuresand its workflow management capacity is also enhanced.
Census India: Use of ICR TechnologyWhat is ICR Technology? Intelligent Character Recognition (ICR) software converts hand printed characters to a machine readable format. This is a very crucial technology which has a high utility value. The ability to recognize hand written characters makes it very suitable and apt for the data processing activities involved in a census and helps in saving a lot of time besides raising the accuracy levels. This could not have been achieved if done manually.The software for ICR is actually based on the science of neural networks which is like a human brain. It is termed intelligent because it is able to tackle the variations in the character shapes.Basics of ICR TechnologyNo two persons can write characters in a similar manner and hence understanding and interpreting the patterns of human writing is more complex and challenging than converting simple machine formats. Variation in characters can occur due to the differences in environment, mood or even stress. A person will fill out the form each time in a different manner (hand writing). Variations will even appear within the same word, depending on where a character appears. The hand written characters are also never evenly spaced across the page making it all the more difficult for recognition systems to effectively break words into their component characters.ICR is able to do all this and thus helps in not only saving a lot of time but does it with enhanced efficiency and accuracy. India is the only country in the world to apply this technology to the massive census exercise.Truly the census 2011 in India is not only a technological marvel but a technological challenge as well.