Innovation is a hallmark of Statistics SA. It makes the difference in the art of continuous improvement, building internal resilience, execution, providing customer satisfaction with better service delivery every time. South African statistician-general Risenga Maluleke has delivered a multimode digital census in SA. The fourth census of postapartheid SA was the first of its kind — a digital census. It was undertaken amid a global challenge of a pandemic Covid-19.
The census had a high undercount. Many commentators have written about this. However high the undercount might be, does not diminish the validity of the count. This is because techniques of compensation through an independent post-enumeration survey (PES), are deployed to scientifically establish the extent of the undercount and adjust for it. When an undercount is less than 5% there might not be a need for adjustment, but once it is 5% or more, it is prudent to adjust.
But it is not only the undercount that gives credence to the numbers. There is in addition the depth of intellect in Stats SA in the form of deployment of Zipf. The Zipf technique provides a predictive gravity model in the Central Place Theory. Zipf, enabled my considered decision as the statistician-general to take an informed position to agree to space the census to a 10-year interval when the minister responsible, Trevor Manuel, placed this forward for formal consideration in 2012. The review of the Statistics Bill, under current consideration, will finally take a formal decision on whether the census is, by the Statistics Act, a five-yearly or a 10-yearly undertaking.
Leapfrogging to a wholly digital census prompted me to look back. I was asked by the United Nations Statistical Commission to present a keynote address in New York on February 24 2012. This is how my keynote went: on October 31 2011, the United Nations announced that the world had reached a seven-billion mark. Population censuses or an account of the population at a point in time within a specified geographic space has a long history. Censuses of population that include housing as a feature are a more recent phenomenon. Earlier censuses covered extremely limited topics and often at that considered a subset of the population — men to deploy to war. They largely followed an assembly method. In the main, census undertakings can be spaced in three periods. This classification is very crude but generally follows purpose of the undertaking, scope of content and coverage, and technique of enquiry.
Evidence suggests that countries classified as in the developing world today have the longest history of censuses. Among the countries that undertook censuses in the period before the birth of Christ are the two most populous, namely China and India, though the time period between the two undertaking a census is measured in thousands of years. China about 5000 BCE and India 300 BCE. The paper attempts to speculate and offer an answer what the future of population and housing censuses, particularly in developing countries, will be. It concludes by saying that the basic undertaking has been and remains moving flesh as in people, wood as in paper and more recently tools as electronic devices replace paper. Internet remains the major solution but remains inaccessible, therefore it is still a solution too distant.
Censuses are the forerunner activities to the creation of national statistics operations and statistics institutions. A census is the fact finder of the nation and a barometer of social progress. It unifies humanity in incredibly special ways. For instance, the most spoken of Millennium Development Goals (MDGs) derive their measures from these undertakings. A census makes and or breaks chief statisticians. The morgue of census takers is full of former chief statisticians.
It is interesting that even as we count the seven-billionth person, the technology of counting has remained virtually the same over the 7,000 years.
Census is about moving mass, whether it is dead trees in the form of paper, flesh in the form of human beings, or tools in the form of electronic devices. The question is, can census takers transit from moving mass to moving bits and bytes of information from point A to B. Evidence suggests that in the majority of jurisdictions the toil of census takers remains the physical porting of mass between points. That mass has remained the same over 7,000 years. It has only become slightly lighter and portable when writing moved from stone to paper, more problematic but more accurate when assembly methods changed to house-to-house enumeration, and better still more efficient when electronic devices were used instead of paper questionnaires.
What are the macro questions?
To answer this, one has first to understand the motive for undertaking a census in temporal and spatial terms. Second, an understanding of techniques of its undertaking and developments that have taken place over time and finally what these behold for the developing world in contradistinction with the developed world. To do so I will draw from census taking across the world and weigh these practices against what the developing world has done or not done.
Motive force for undertaking a census
Unique circumstances over time influenced the purpose, and to some extent the content, of the census. These circumstances differed by milieu. In ancient times, the compelling reason was one of necessity for acquiring the fiscus through conquest. This motive force covered 6,500 years from 5,000 BCE to the 1500s. Between the 1600s and 1800s, the motive force for censuses was largely based on mercantile and industrial revolution interests which include colonialist expansions. By the turn of the 1900s to date we have noted a shift in the motive force for undertaking a census. The shift has been towards meeting basic needs of society and undergirding activities of modern societies in a sustainable fashion.
It is interesting that even as we count the seven-billionth person, the technology of counting has remained virtually the same over the 7,000 years. In the main it is about the physical act of moving mass and mobilising mass on a grand scale. One extreme prospect for radically changing the future of censuses in the world but particularly for the developing world is apparent. This radical change is in the logistics of literacy, access and ability to move bits and bytes of data over the electronic net. However, the prospect of such change, while already in existence in the Scandinavian countries and in parts of Asia, as in Singapore, remains remote, especially in the developing world. The scenario presently suggests that the seven-thousand-year-old practice of moving mass be it paper, or electronic devices and people will sustain for a long time. What then mitigates the future of census taking particularly in the developing world?
The micro-level questions
Having addressed the strategic question or the macro level question of the motive force, the project has to be defined in terms of the practical ways of executions. In what way would the future of census taking matter when for almost 7,000 years populations have been counted and reported upon in pretty much the same way, so why not stick with the known and tested? (i) Should the future then matter in terms of expansion on the content of the questionnaire and coverage of subjects? (ii) Should it matter as far as improvements in the technology of mapping, collection, processing and dissemination are concerned? (iii) Should it matter in relation to how the census is then defined given developments as seen in France and more recently in the US with the advent of a rolling census? (iv) Should it matter in relation to developments as have been observed in the Scandinavian countries, where the management of administrative records have been at the core of governance and the count is about interfacing a variety of databases? (v) What are the implications for privacy of information and safety of citizens before the state?
Questions to be answered
(i) Should the future then matter in terms of expansion on the content of the questionnaire and coverage of subjects?
The debates on how long the census questionnaire should be remain a great curiosity, and census practitioners have found a way of resolving the debate. In instances they have stuck to about 75 questions for the entire population. Fifty of these have been at the individual personal level, such as sex, age, education, employment and so on, and 25 have related to housing and household-level concerns, such as electricity, sanitation, water, type of materials for housing. In other countries census practitioners have adopted the long and short form options, with the long form being a sample of 10% in most instances but contains all the short-form questions.
A matter of professional controversy and distress in the short-long form saga relates to the Canadian census of 2011, where the time-series, a brains trust and capital for any statistical organisation, was compromised by making the long form optional. It will be a matter of academic inquiry for a long time to see how the time-series was violated by this professionally barren decision and how possibly it gets corrected. In Africa in particular, the form remains long and includes aspects of agricultural activities, and in instances covers livestock. In the case of the Sudan in their 2008 Census, they used both the long and short form. SA used the long form only and experts and practitioners always rage like bull terriers when they fight over which questions to include. The matter of short form looks very distant, and for a long time in the future SA may stick with only the long form.
Challenges to the developing world
What is important on the question of content of the questionnaire, whether short or long, is consistency with content, as well as compulsion to answer questions. In fact, in the Sudan, the long and short questionnaires were executed in the 2008 Census very successfully. The question is whether the Sudan, which now consists of two countries, will preserve this time-series while responding to new and emerging needs but do so following sound statistical practice consistent with the fundamental principles and in particular principle 1 and 2, rather than political convenience as dictated to in the case of Census 2011 of Canada. Lessons from Sudan become critical for the developing world on this front.
(ii) Should it matter where improvements in the technology of mapping, collection, processing and dissemination are concerned?
The advances in technology have eased the burden associated with census taking, albeit painful experiments and monumental failures accompanied adaptations to these technological advances. The 2001 Census of SA flirted with scanning and serves as a case in point of difficulties related to use of technology when the skills are deficient, so is the case of Ghana in the 2000 Census and Kenya in 1999 Census. A lot of insight should emerge from the harrowing and career-threatening experiences of officials leading these processes. Today, however, the scanning technology is easier to use unlike in the 1970s. In fact it is so that at the turn of this century when they re-emerged, they proved to be a great solution.
The Australian Bureau of the Census has been using scanners for an exceptionally long time and notched immense successes in the 80s already. The UK and the US adopted the technology in the 2000 round of censuses, and Canada has done so in 2005 with a host of other developing countries doing so in the first five years of the 2010 round. That vintage includes Africa’s giant, Nigeria and Sudan. In mapping, aerial photography achieved dramatic improvement from sketch maps with the rapid advent of digital imagery. Cartographic labour was eased, and the quality of fieldwork has improved.
In data processing, the advent of scanning technology has brought about improvements in the quality of data processing and in particular in the categories of questions such as occupations and industry. These technologies have also improved the speed of processing especially for mark-reader based instruments. But not only that, improvements are visible as well in character recognition. The latter is based on the intelligence built in to the dictionary engines that interpret the handwriting. Cellular phone technologies have made reporting easier and hold promise for real time work reporting and resource deployment to resolve problems instantly. The 2011 Census of SA also applied cellular-based reporting rather successfully. Other hand-held devices with inbuilt global positioning systems (GPS) have now emulated the US TIGER files. This was a state-of-the-art technology from the 80s up to the turn of the century that the US Bureau of the Census prided itself in and indeed became the world leader and contributed dramatically to the way census data could be disseminated. These hand-held devices have replaced the scanners, and Brazil bears testimony to the efficacy of these devices and their general application. The Brazilian approach affirms that these technologies are more amenable for more inaccessible environments. Yet another technology that holds the best promise of all is the internet as the future nirvana of facilitating a massive national conversation. Countries such as Canada and the US have recorded interesting and improving results in terms of what emerges out of internet-based enumeration. However, the US could not cross the Rubicon in the 2010 round to adopt the internet as a method of enumeration among a battery of many other methods adopted for data collection.
The technologies as discussed above appear to hold great prospect for developing countries, where physical infrastructure is non-existent and is in extreme disrepair. A census undertaking requires roads so that census materials can be moved, requires buildings for storing materials as well as for providing training to teams of enumerators. The technologies, and in particular the electronic devices, appear to answer all the questions that the Democratic Republic of the Congo (DRC), Angola, Afghanistan, Somalia and many other developing countries face as imposed by the census challenge. While the South Americans, Mexico, Brazil and Chile, with the advances they have made in geography, tended to pave the way in census collection and dissemination technologies, they have also led the way, in the case of Brazil, in the use of hand-held devices for data collection. These devices, while already in use in many countries in small-scale surveys, have not been used in censuses by most.
The conservative statisticians and census takers appear to have been too risk-averse and have been beaten to the game by too bold a step by Brazil in using these devices in their 2010 census. Not only has Brazil used these devices but proved that with amazing speed in results. They have now managed to convince the Cape Verde to use similar devices in their census and are discussing with Senegal and the DRC to use exactly the same devices. If they worked in the Amazon rain thickets, why can they not apply in the DRC equatorial forests? When Brazil took this bold step and demonstrated proof of concept in 2009, I was quite hesitant and not prepared to experiment with this despite my being quite prone to adventure. It was perhaps the scars from scanning technology in 2001 that held me back. Persuasion from dear brother Nunnes, whom I trusted, could never lead me into a dark alley despite my trusting him. But neither was their neighbour to the north, the US, prepared to take such a chance with the hand-held devices. Brazil has shown that these devices work, and they have the power of geo-positioning too.
As far as internet-based surveys are concerned, Korea conducted research for their 2005 Census with the aim of introducing internet-based census of the population, particularly for the single-person households. In that census, they had a 0.9% penetration. Their hope is that they will reach 30% of the population in 2010. The 2005 test showed that there are material gains in the use of internet. In fact, it is clearly demonstrated that this results in better quality information compared with paper-based collection. This is because of the interactive logic built in to the devices. Canada, Singapore, Norway and Australia raised the bar quite high in their plans for online census questionnaires, and they also expected that 30% of their respondents would be internet based. Daniel Castro, a senior analyst for the Information Technology and Innovation Foundation (ITIF), in his submission in 2010 chastised the US Bureau of the Census for not following the example of these countries, though earlier results in the 2000 round showed to them that the internet could be the way to go. In his submission he encourages the bureau to do so in the 2020 Census. We shall still hear how these pioneering countries fared with internet in the 2010 round of censuses.
My successor, Maluleke, took the advice of Daniel Castro to heart, ahead of the US Bureau itself. He did so amid the adversity of Covid-19.
Dr Pali Lehohla is the director of the Economic Modelling Academy, a professor of practice at the University of Johannesburg, a research associate at Oxford University, a board member of Institute for Economic Justice at Wits and a distinguished Alumni of the University of Ghana. He is the former statistician-general of SA.
Would you like to comment on this article?
Sign up (it's quick and free) or sign in now.
Please read our Comment Policy before commenting.