What’s next for official statistics

While the pandemic underlined the importance of data and statistical offices, it also exposed limitations and challenges related to data collection – for example, slow data acquisition, data gaps, and inconsistent definitions across different jurisdictions and economies. Therefore, to become truly fit for the future, statistical offices must modernize their structure and processes and standardize data collection, analysis, and coordination.

How technology is changing the role of statistical offices

Historically, the role of the statistical office has revolved mainly around coordinating household surveys and censuses. Today, that is changing. The power of Big Data and artificial intelligence (AI) has disrupted how we collect and process data. However, many of the same mathematical and statistical methodologies are still applied for analyzing the data.

As the volume and complexity grow, statistical offices can utilize AI technologies, such as machine learning and deep learning models, to uncover insights that might be difficult or impossible to find using traditional methods. Furthermore, by using secure data exchange mechanisms, statistical offices can set up online data channels with external authorities, enabling them to keep their data up to date in real time. This leads to a shorter time lag between an external request for statistical analysis and the result.

A great example is an e-census, where clicking a button can produce results within just a few hours – a critical advantage, as we saw from our work on the Oman e-Census 2020. In addition to improved speed to market, digitalization of data sources and online data processing ensure greater accuracy of the underlying data. Since automated data processing starts from the original data source, the statistical analysis is more consistent and less prone to interpretation errors.

Statistical office of the future

In this light, specific characteristics will allow a statistical office to be future-ready. Here, we have outlined the most important.

Adaptable and agile

The ability to adapt to innovative technologies, data sources, and analytical methods will be critical to ensure that statistical offices can continue to supply relevant and timely data to users. Yet, to enable adaptability across the statistical ecosystem, statistical offices need to become agile and respond quickly to new data sources, emerging trends, and the changing needs of the users. Agility will foster a willingness to experiment with new data sources and methodologies and be more cost-effective and quicker as digitalization and online data processing allow for faster production and dissemination of the results.

Collaborative

In the past few years, we have seen an increasing number of data sources as data is collected from everywhere by both public and private data providers. In this massive “data market,” collaboration with other agencies, data providers, and user groups can significantly support statistical offices in identifying and validating data quality and developing new insights into social, economic, and environmental issues.

One case study on effective collaboration is Colombia. The National Administrative Department of Statistics in Colombia, PARIS21, and World Data Lab joined forces to support innovation by using cutting-edge technologies and supplementing existing survey and census data with geospatial data to generate up-to-date insights on demographics and poverty estimates. This allowed public entities to formulate and implement policies focused on the most vulnerable in society.

Collaborative partnerships can help statistical offices overcome resource constraints, share expertise, and leverage technology and data assets more effectively. However, with great power comes great responsibility. Concerns around data privacy and data stewardship remain a question. Statistical offices must adopt ethical and responsible data practices to ensure data is accurate, reliable, and compliant with local regulations. They must also increase transparency to ensure data is relevant and accessible to the broader society.

Finally, collaboration with users will be imperative to understand user needs better and produce relevant and valuable insights. For instance, Eurostat has launched a user satisfaction survey to understand its users’ needs, while Statistics Netherlands has developed a user-centered approach to data dissemination.

Innovative

Modern statistical offices will be driven by innovation and outcomes. They will utilize the latest technologies and systems to transform data collection and analysis. By combining official statistics with innovative data services that are crowdsourced, immersive, and insights-driven, they will also be able to maximize value for users and decision-makers.

Moreover, visualization through dashboards will be crucial for engaging users with analytics, and storytelling will help contextualize the data to make its implications easier to understand, enabling faster and better decision-making.

Conversely, this will also require a culture of continuous learning and improvement, where statisticians are encouraged to develop new skills and stay updated with emerging data analysis and analytics trends.

Transforming the way we understand the world

With the power of AI and secure data processing technologies, statistical offices can take their data analysis to the next level, uncovering new insights and transforming how we understand the world.

Modern statistical offices have some key characteristics that allow them to be future-ready. By becoming adaptive and agile, they can respond quickly to new data sources, emerging trends, and the changing needs of the users. Collaborating with other agencies and users will help them develop new and better insights, while innovation will make data more accessible and easier to understand.

By identifying new opportunities and challenges proactively rather than reactively, modern statistical offices can immensely impact economic development, environmental issues, and the wider society. In the case of Colombia, innovation in statistical methods, collaboration, and cutting-edge technology made it possible to track and measure impact in real time and, therefore, enhance decision-making.

Now is the time for statistical offices to embrace technology and step into the future fully by leveraging the abundance of data from the pandemic.

Find out more about our real-time economy solutions here.