Facebook uses scanning technologies, alerts authorites about content

In March of this year, authorities in south Florida arrested a man in his thirties who had used Facebook to make plans to meet up with a minor. According to Reuters, a program designed by the social networking platform to monitor suspicious communications between adults and minors led to the arrest. Facebook regularly scans user content for criminal activity, but the monitoring program is something the social media giant has generally kept quiet about. Reuters explains, “Facebook generally avoids discussing its safety practices to discourage scare stories.”

Though often hidden from view, this monitoring program is one of the most advanced of its kind. CNET describes the general mechanics of the program:

Facebook's software focuses on conversations between members who have a loose relationship on the social network. For example, if two users aren't friends, only recently became friends, have no mutual friends, interact with each other very little, have a significant age difference, and/or are located far from each other, the tool pays particular attention.

The scanning program looks for certain phrases found in previously obtained chat records from criminals, including sexual predators.... The relationship analysis and phrase material have to add up before a Facebook employee actually looks at communications and makes the final decision of whether to ping the authorities.

According to Reuters, this sort of scanning is commonplace for platforms like Facebook—most large social media companies scan chats for inappropriate language and exchange of personal information. However, many social media platforms--especially those tailored for younger audiences--walk a tightrope between utilizing these tactics to safeguard against illegal activity and providing a less restrictive social media platform that will engage users. Reuters explains:

From a business perspective, however, there are powerful reasons not to be so restrictive, starting with teen expectations of more freedom of expression as they age. If they don't find it on one site, they will somewhere else.

Scanning users’ content is not new terrain for Facebook. In April a document leaked showing the kinds of user information Facebook releases to authorities when subpoenaed. While the document provoked public backlash, Facebook is clear about what it does with users’ information in the “Information for Law Enforcement Authorities” section of its website.

Facebook acknowledges the difficulties inherent in monitoring content on its platform for criminal activity. Facebook’s Chief Security Officer Joe Sullivan, speaking to Reuters, explains, “We've never wanted to set up an environment where we have employees looking at private communications, so it's really important that we use technology that has a very low false-positive rate.” Digital Trends explains that Facebook takes active measures to limit communication between minors and unfamiliar adults on its site. Examples include not listing minors within the Facebook search tool and allowing only direct friends of minors into a direct chat. It then scans communications that do take place for keywords and patterns derived from an analysis of chat logs taken from prior criminal cases in order to identify possible threats.

Scanning technologies of this design are just beginning to come to the forefront for various websites. While some sites are reticent to embrace them fully, fearing revenue loss, this example highlights the tactics used and suggests a possible upward trend in surveillance by social media platforms.