More Than a Number: China’s Social Crediting System Comes with Inherent Bias

Introduction

In the episode “Nosedive” of the dystopian science fiction television series Black Mirror, Lacie, a woman obsessed with climbing up the social ladder but unable to secure the apartment of her dreams because of her low social media score, follows the advice of a social media consultant by attempting to impress “High Fours,” people whose score is closest to her society’s standard of social achievement, the five-star rating. Conceived by Black Mirror creator Charlie Brooker as “a satire of acceptance and the image we like to portray”, “Nosedive” speaks to present social media anxieties and our excessive adoration of social media influencers. But to actually lose one’s job because of a low social rating, as the episode portrays, is far-fetched, right? Not exactly. In fact, this is exactly what may be happening in China today due to its social crediting system.

By the Numbers

China has the strongest presence on the Internet in the world, with nearly one billion users. China also has some of the strictest media censorship, and most of the websites and Web 2.0 services used in the Western world are blocked completely and subsequently replaced by “copycats” of their own: while the Western world is used to Google, the Chinese have Baidu. Instead of Twitter, the Chinese have Weibo. In place of Facebook, the Chinese have created Renren. Instead of viewing videos on YouTube, Chinese people view them on Youku and Tudou.

By keeping Chinese people on their own websites that usually record and store all their personal data within buildings in the country’s capital, Beijing, the government can constantly monitor what is done online and by whom. And now that China is implementing a national social crediting system, this government surveillance coupled with the Chinese people’s embrace of their censored Internet could lead to a disastrous result. History has shown that people in power -- especially those with access to innovative technology -- tend to favor the wealthy and elite.

Social Credit

So, what is the new Chinese social credit system and how does it work? Formally introduced in 2011 by Chinese Premier Wen Jiabao, the social credit system is the Chinese government’s project to create a unified record system to track and evaluate its citizens, businesses, and regional administrations for trustworthiness. Some regional administrations rate trustworthiness using a numerical value, while the central government uses mostly blacklisting and whitelisting. It was officially set to begin in 2020, but the deadline was pushed back because of the COVID-19 pandemic. Some aspects of it are similar to the credit checks that banks run--using credit rating companies such as Experian in the United States--before providing someone with a loan. It also resembles the way that social media value is measured in the West, which usually depends on an Internet user’s follower count or the amount of likes they receive on online posts. The Chinese system, however, will apply these ratings to all aspects of Chinese life. A citizen or business can be given a high ranking for donating to charity, or have value deducted for grievances such as playing music loudly. 

Once the national system is fully implemented, Chinese citizens, businesses, and local governments will be assigned a number based on the data gathered from Chinese “big tech,” that is, the innovative technology across the country that collects personal information as well as other data gathered from websites that people and small businesses may access.

One such technology key to information gathering is Artificial Intelligence (AI) that uses facial recognition technology to monitor citizens and their movements. Since the transgressions that the Chinese government will look for can range from jaywalking and unsound business practices to evidence of sharing anti-government rhetoric, once the system is in place, AI and the cameras will be able to do most of the work in tracking the citizens and alerting the government of violations.

The negative impact of a bad score for businesses and people will also varies. As mentioned, the central government already has a running blacklist of businesses and people who they feel have acted poorly. By being blacklisted, the government publicly shames the transgressors while banning some personal rights and potentially limiting their ability to continue to do business. Some local governments have offered positive incentives for having a good score to try to increase trust in the system, such as prioritized healthcare and deposit-free housing.

China and the Internet

As stated earlier, the Chinese government blocks Western social networks such as Facebook, Twitter, and YouTube, replacing them with their own. By allowing only these websites to exist and monitoring the content their citizens are allowed to post, the Chinese government can control the narrative and opinions around the country.

Say something about a censorship law or something damning about your official? One can be expected to be taken offline, prosecuted, and even in some cases permanently blacklisted from basic human rights such as access to healthcare and transportation. Say something good or release a story about an official the government may already want ousted? Expect to gain some notoriety when the government allows your opinion to be circulated for days before they finally take it down. This type of selective censorship may raise alarms to a Chinese citizen who now faces the threat of the government applying scores that influence opinion on who is worthy of access to basic human liberties.

With China being the most populated country in the world, holding about 20% of the world’s population according to their last census, the public’s desire for a watchful eye in the form of social crediting is expected, but may also be skewed because China is an authoritarian regime:  

Overall, [the Chinese] report a high degree of approval of SCSs [social crediting systems], with 80% of [participants] either approving or strongly approving SCSs. Only 19% of [participants] perceive the SCS in value neutral terms (neither disapprove nor approve) while just 1% reported either strong or somewhat disapproval. To some extent, the high degree of approval of SCSs and the almost non-existent disapproval we found might reflect the nature of conducting a survey in an authoritarian setting—while were clearly informed that that the data were anonymized and to be used for research purposes only, some more cautious [participants] may have falsified their preferences to a degree due to concerns about expressions of disapproval resulting in reprisals from the state. 

This same ideology may contribute to the public’s general acceptance of their censored Internet. As Chinese people do not get the option to speak out against the government, one must hope that the new implementation of technology will cause more good than harm, especially now that the Chinese government is pushing the surveillance of children, as illustrated in the Wall Street Journal’s special report “How China Is Using Artificial Intelligence in Classrooms." At one point, the report focuses on an electroencephalography (EEG) headband that young students must wear to gauge their attention during lessons, whose output is monitored by the teacher and shared with all the students' parents. When the reporters show the footage to neuroscientist Theodore Santo of the University of California, San Francisco, he is surprised, noting that the technology is susceptible to false readings, so he is not sure how useful it would be for measuring attention. The Chinese teachers, however, explain that the use of the headbands themselves have "forced to make students more disciplined." In other words, the technology is molding the behavior of students at a very early age.

Xinjiang

Xinjiang in China. Xinjiang means "new borderland" and borders the countries of Mongolia, Russia, Kazakhstan, Kyrgyzstan, Tajikistan, Afghanistan, Pakistan, and India.

Furthermore, China has long used technology to monitor and suppress their ethnic minority populations. In Xinjang, a region on the border of China and home to the country’s largest Muslim population, adherence is instilled in the forms of virtual checkpoints and the swiping of ID cards. The Chinese government has completely digitized policing and is creating a state of constant surveillance to keep the population in line. Some people are allowed to bypass these checkpoints and are not subject to digitized policing. Chris Buckley and Paul Mozur of the New York Times report that “at many checkpoints, privileged groups — Han Chinese, Uighur officials with passes, and foreign visitors — are waved through “green channels.” Although the Chinese may seek more trust and policing, this is an obvious form of targeted surveillance that can have the possibility of trickling into more minority groups.

Conclusion

With China already implementing its social crediting system and the fear of being blacklisted at the forefront, I understand that resisting the government is a daunting task. I also understand that this is a great moral issue in which an increased number of Chinese people will be stripped of their basic human rights. Technological innovation is growing at a faster pace than anyone can keep up with and blindly following the government’s application of said technology will prove disastrous for all parties, but as history has shown, the minority will always be left with the biggest burden.

I implore the Chinese people to resist the use of such a credit system. The first step would be choosing not to opt-in to the private aggregators of data. Before the mass implementation of the social crediting system, private companies have begun to gather information on an opt-in basis, I implore you to resist. The government wins every time one of their allotted big corporations is favored. Small businesses and anyone with hiring power should also disregard consideration of the crediting scores when making hiring decisions. By doing this, the people could nullify the crediting system and leave it to the government to use it solely. Though Chinese citizens might think that the benefits of a surveillance state (including safety and security) outweigh the harms outlined in this paper, I ask you to reconsider. Further, I argue that the risk of loss of human rights and liberty for minor transgressions such as jaywalking should outweigh any benefits.

At the end of “Nosedive,” Lacie is finally overwhelmed by her lifelong desire to secure a high score. She realizes that her desire to fit into the pastel-perfect life of her disingenuous friend is made up of a societal standard that will never allow her to truly live a fulfilling life. Within a day, she was heaved to the bottom of the social ladder, along with the very people she ignored and chastised her entire life. By drinking out of a red flask of whiskey that finally released her from her flawed perception,* she was able to finally live again and see the people for who they really are, past their assigned scores. I ask Chinese citizens to do the same: take the red flask and see past whatever social crediting system may be implemented among them. Your children, your neighbor, and yourself will thank you in the long run. We are more than a number.

Notes

*This is reference to a scene from the 1999 sci-fi film The Matrix, where the protagonist is given the choice between learning the hard truth by taking a red pill or living in a comfortable illusion by taking a blue pill.

Further learning

Antonia Hmaidi. "The" Social Credit System in China. 35C3.

• Antonia Hmaidi. "'The' Social Credit System in China." (Video). 35C3. 27 December 2018. A lecture for the 2018 Chaos Computer Club Congress that seeks to provide a more nuanced view on the Chinese Social Credit System than what the Western media offers.
• Victoria Adelmant. "Social Credit in China: Looking Beyond the 'Black Mirror' Nightmare." Center for Human Rights and Global Justice, New York University School of Law. April 20, 2021. A discussion of the human rights implications of the Chinese Social Credit system. With many additional resources and explanations of the organizations involved.
• Eunsun Cho. "The Social Credit System: Not Just Another Chinese Idiosyncrasy." Journal of Public and International Affairs. 1 May 2020.

Discussion questions

• What are forms of financial, work performance, and social credit scoring in the United States? How are they similar to the Chinese system? How are they different? In your opinion, could the institutions that run our financial and our social credit scores be turned into a loosely unified system similar to that of China?
• What kind of citizen is the Chinese government attempting to obtain through its system of credit rewards and punishments? What could be the reasons for making every citizens' score public, accessible, and displayable? What could be the reasons for deploying monitoring technologies in school?
• What could be some unintended consequences of manipulating a population's social behaviors?