ONI Methodology, Tools, and Data FAQ

Q: What methods does the ONI use to investigate Internet filtering practices?

A: ONI employs a multidisciplinary approach that includes technical enumeration of state mandated Internet censorship policies and practices, field research undertaken by regional and country-level experts as well as legal and regulatory analysis. This mixed-methods approach combines technical data with contextual indicators to give a richer view of national level filtering regimes than would be possible from technical metrics alone.

Q: How does ONI determine if a website is filtered?

A: ONI conducts technical testing of Internet filtering using a software system developed to identify blocked URLs. This system is based on a client / server architecture. The software client is distributed to researchers and volunteers within countries of interest to ONI. The client queries pre-defined lists of URLs which are accessed simultaneously both in the country suspected of filtering and a control country (e.g. Canada) which does not filter the type of content for which the ONI tests. The data that is collected includes URL, network errors, IP address of the URL, DNS settings, DNS entries, HTTP codes, HTTP headers, page body, and in some cases trace routes and packet captures. Where appropriate, tests are run from different Internet service providers (ISPs) and across multiple days and weeks to control for normal connectivity issues. The resulting data is then transmitted to a server housed at the University of Toronto.

The results from within the target country (field) and the control country (lab) are compared through both automated and manual analyses to identify what content is filtered and the method of filtering. These methods of filtering (PDF) can range from proxy filtering which displays a transparent notice that content is not permitted (a ‘block page’) to other forms of connectivity disruption which resemble innocuous network errors. Filtering patterns are determined for each ISP which engages in filtering and these patterns are used to automate the analysis of future results.

This methodology has a number of benefits: it is a generalized testing method that has been successful at detecting censorship in a large number of different countries; it is able to collect rigorous data that is representative of the general user experience of citizens in the country. This methodology also has limitations: data collection and analysis processes can be slow; the process relies on “real” users (however this requirement also contributes to some of the benefits enumerated above); Internet filtering is dynamic and takes on different forms and generalized testing methodologies may not fit all possible use cases.

Q: Is ONI’s testing software available to the public?

A: Currently the software system ONI uses to test for Internet censorship is not available to the public. ONI’s testing system was developed for use by a small group of researchers, and was not designed to incorporate testing contributions from the general public. It is a research tool, not software intended for public use. Historically, ONI has been concerned with the potential for the testing software to aid censoring regimes in their filtering practices and has thus not made the software publicly available. We are exploring alternative methods and tools for studying Internet filtering. We also continue to assess the risks and benefits of different approaches.

Q: What kind of content does ONI test for?

A: In order to identify Internet filtering, ONI checks two lists of websites in each of the countries tested: a global list (constant for each country) and a local list (different for each country). The global list is comprised of a wide range of internationally relevant and popular websites including sites with content that is perceived to be provocative or objectionable. Most of the websites on the global list are in English. The local lists are designed individually for each country by regional experts. They have content representing a wide range of content categories at the local and regional levels, and content in local languages. In countries where Internet censorship has been reported, the local lists also include many of the sites that are alleged to have been blocked.

These lists are samples and are not meant to be exhaustive. Content on these lists is grouped into one of four broad themes:

• Political (This category is focused primarily on Web sites that express views in opposition to those of the current government. Content more broadly related to human rights, freedom of expression, minority rights, and religious movements is also considered here.)
• Social (This group covers material related to sexuality, gambling, and illegal drugs and alcohol, as well as other topics that may be socially sensitive or perceived as offensive).
• Conflict/Security (Content related to armed conflicts, border disputes, separatist movements, and militant groups is included in this category).
• Internet Tools (Web sites that provide e-mail, Internet hosting, search, translation, Voice-over Internet Protocol (VoIP) telephone service, and circumvention methods are grouped in this category.)

Q: Where can I find a list of all the websites that are filtered in a country?

A: ONI country profiles report on the content filtered in each country, including both the type of content found to be filtered and examples of specific websites that are filtered. As ONI testing lists represent only a sample of websites, they are not exhaustive lists of all the content which may be filtered in a country. In addition, the technical filtering data alone does not provide a complete picture of Internet censorship and content regulation in a country. ONI country profiles add contextual information to technical testing data to provide a more complete view of each country’s filtering regime.

ONI has historically not published complete raw testing results for a number of reasons, including ensuring the security of testers and to prevent ONI testing data from assisting censoring regimes. ONI continues to assess these concerns and is currently reviewing its data release practices.

Q: What ONI testing data is available for download?

A: ONI has released summarized data of its global testing results in CSV format under a Creative Commons license here.

Q: Does ONI test for Internet filtering in real-time?

A: ONI does not conduct real-time testing of Internet filtering. ONI tests are snapshots of the accessibility of a sample of content at particular points in time. Therefore, results of a given country are accurate for the sample of URLs tested at a particular point in time and may not reflect the most current state of accessibility. ONI results, including the summarized global testing data and country profiles, indicate the time period in which testing was conducted.

Q: Does ONI testing (or other's testing) provide a comprehensive picture of Internet filtering worldwide?

A: ONI tests provide a snapshot of accessibility to a limited subset of the Internet for a limited number of countries. There are many other projects which complement the picture provided by ONI research, including Herdict, Google Transparency Report, Chilling Effects, Measurement Labs, Tor Metrics, Open Observatory of Network Interference, and many others. Each provides a different view into the ecosystem of information controls, but none alone can provide the full picture.

Q: Why are there no results from [country x]?

A: While ONI has conducted testing in 74 countries, it is not a comprehensive view of all global Internet filtering and thus not all countries are included. ONI does report on Internet filtering worldwide through the OpenNet blog. If there is a country that ONI does not have data or reports on that you feel is important please contact us at contact [at] opennet.net

Q: Why does ONI report that there is “no evidence” of censorship in a country in which there have been reports of censorship? What about countries that engage in Internet surveillance, intermediary censorship, intimidation of bloggers, etc.?

A: ONI ratings of filtering are based solely on the results of the empirical testing process described above. ONI cannot test in all countries on a regular basis and thus some results are based on testing data from prior years. The year of testing is indicated in both the country profiles and summarized filtering data. In addition, while there are other forms of information controls beyond Internet filtering these are not included in the filtering ratings.

Q: How does ONI evaluate filtering on private or institutional networks?

A: ONI testing seeks to capture the experience of an average Internet user, and thus generally tests on consumer ISPs without the use of circumvention tools. For many who access the Internet through a work or school network, for example, there may be additional layers of filtering. ONI results are intended to reflect the depth and breadth of national filtering regimes and thus do not take into account private or institutional filtering.

Q: Why does ONI identify Canada and the United States as countries with ‘no evidence’ of filtering when it is known that both engage in filtering of content relating to child exploitation?

A: The ONI testing server is housed at the University of Toronto and thus is subject to Canadian laws limiting the access of content relating to child exploitation. As a result, we are prohibited from attempting to access this content and cannot test if it is blocked. We have thus excluded child exploitation content from our evaluation of global Internet filtering. However despite our inability to engage in technical testing for this content, the ONI has done studies on the issue. For example, ONI has produced an extensive regional overview of North America that analyzes the legal and regulatory frameworks around content filtering and child exploitation in Canada and the US. Additionally our second edited volume Access Controlled: The Shaping of Power, Rights, and Rule in Cyberspace includes a chapter (PDF) by Nart Villeneuve that provides a historical overview of online child pornography controls and examines the range of policy responses that have been employed.

If you have additional questions or comments please contact us at contact [at] opennet.net