Approaching Internet Fact-Checking Via Crowdsourcing

Abstract

With the fading popularity of the newspapers, the internet has been the primary source of information for many. People have many sources on the internet that they get their news from. The increased affordability of internet services has led to a great rise in the number of daily internet users. In India alone, the number of internet users has gone up by approximately 146% over the past 5 years. While online newspapers were initially popular, people have taken to alternate sources, the biggest being social media, to receive their news. This poses a serious problem; the possibility of fabricating information and circulating it among the masses as legitimate news from an authentic news agency. This has led to a surge in the amount of \'fake news\' people consume on a regular basis. The circulation of fake news has very serious effects, even some as big as affecting election outcomes. The research in this paper is focused on addressing the problem of identifying fake news on the internet via the means of crowdsourcing. Crowdsourcing is the process of obtaining information from a large number of people, typically via the internet. Previous research on the topic has given a standard model for fact-checking via crowdsourcing. This paper aims to address the demerits of previous research papers and improve upon them so as to make crowdsourcing a reliable method of fact-checking in an effort to curb the spread of fabricated news on the internet.

Introduction

I. INTRODUCTION

The rise in popularity of social media and the number of its active users has led to a lot false information being circulated among the masses as legitimate news from credible sources. This has very serious consequences as discussed ahead in the presentation. One method of tackling this problem is fact-checking. Independent groups exist with the sole purpose of verifying news on the internet, but there is only so much ground they can cover. Crowdsourcing, a method of obtaining information via netizens, is a good alternative to independent fact-checking groups.

II. RELATED WORK

The proposed process in this paper [1] is based solely on voluntary human efforts, but can set the base for developing ML algorithms. Merit is that it finds a method to deal with high volumes of information. The demerits including encountering cases of confirmation bias as well as having no means of verifying the legitimacy of the verifier.

The author of paper [2] focuses on the truthfulness of a piece of news as directly opposed to fake or not. Takes a multidimensional approach to quantifying truthfulness. The merit is that it points out psychological tendencies and flaws when verifying information. However, it doesn't actually check provide a solution for checking if a piece of information is true, rather just how likely it is to be deemed truthful.

The prototype system in [3] uses social argumentation to verify the validity of proposed alternative facts and help in the detection of fake news. The merit is that all the sources and users are rated across many criteria (SourceRating, ContentRating, QuestionRating, etc.). The demerit is that the legitimacy of the moderator can’t be verified.

The paper [4] proposed an approach to address the issue by adding fact-checking flags in the hope that flags could alert people of information falsehoods and stimulate critical thinking. It addresses the problem of confirmation bias. However, this study’s findings are based on concise news items that only include headlines, leads, and accompanying images.

This study [5] focuses on the characteristics misinformation and falsified news in Greece, based on the findings of a factchecking platform that uses  professional fact-checkers in combination with crowdsourcing strategies collaborating with Facebook. The merit is the classification of untrue stories as misinformation, hoaxes, scams, fake, etc.

The author of this paper [6] aims to come up with a model that can accurately predict trustworthiness of online users without a web of trust. The merit is that it does come up with a model that they claim is accurate, however this model isn’t tested against an actual web of trust so it can’t be considered reliable yet. 

III. PROPOSED SYSTEM

 With the increase in active internet users over the past 5 years, the amount of information and news being circulated on the internet via social media platforms like Twitter, WhatsApp, Instagram and Facebook has increased significantly. This has led to a huge increase in false and fabricated information being circulated via these platforms. Fake news often tends to pose as being from an authentic source whose legitimacy goes unverified, thus making the masses believe them more easily. To help overcome this issue, we propose creating a web-based platform with the sole purpose of internet fact-checking via the means of crowdsourcing. Crowdsourcing is the process of obtaining information from a large number of people, typically via the internet. Crowdsourcing helps address the issue of an independent body having large volumes of information to be verified.

The website’s features comprise of:

1. Text/image/video posts
2. Post categorization
3. Comments
4. Rating comments
5. User trust score generation based on ratings

The systematic flow of the system is as follows:

a. The user logs in to or signs up with the webpage depending on their previous activity

b. The user is redirected to the homepage where they can view posts by other users and can also post their own information to be verified

c. Other users can comment on posts

d. Users have a Trust Rating (TR) which is affected by the ratings they receive on a comment

e. Users build up TRs under different categories of posts

f. Certain users with a threshold amount of TR in a category receive a Verified badge under said category

g. The Verified badge appears next to their name on all comments as long as their TR for a category is above the threshold level

h. The Verified badge helps add to the legitimacy of users posting proof on posts

Fig. 1 shows the architectural diagram of the system

 The architecture diagram in figure 1 shows how the members involved in the system interact – Requestor, Participant,

Verifier

• The requestor can post the information that he wants to verify.
• The requestor can search for post with similar information as well.
• The participant adds comments on posts involving proofs such as images, videos, etc

IV. IMPLEMENTATION

The current implementations of the system are as follows:

A. Web Interface

A web interface has been created with ReactJS. It comprises of multiple modules as described below. 

B. Authentication

Google Sign-In via Firebase authentication was emplyed to authenticate users of the platform. Firebase’s server generates auth tokens to grant users access to the platform

1. The user sends their credentials to firebase server
2. The firebase server validates these credentials and generates an auth token if the credentials were valid 
3. Data can then be accessed using this token 

C. Post Creation 

 Any user may choose to post information to be verified on the website if it doesn’t already exist. Every post has a title, description, category, and could also have media attached in the form of an image or video. Posts can be commented upon by other users of the site with proof of legitimacy of the information. A user will need to vote on the different aspects of the post individually as true, false or partially true and also provided a justification of their votes in the form of a descriptive text. Only after the above conditions are satisfied will a user be able to comment on a post. Comments can be rated by other users from 1-5 on the basis of perceived quality

D. Comments

Users can post comments on posts by other users. The comments themselves comprise of a Title Rating, Description Rating, Image Rating (if applicable) and text desciption. The rating parameters are True, False and Partially True. The commenter is expected to justify their ratings for each aspect. These comments can be rated by other users from 1-5 stars, with 1 being the worst and 5 being the best.

E. User Legitimacy

As each post is specified under a certain category decided by the user posting the information. Every comment by a user will be recorded under said specified category. Every rating the comment receives will be saved under ratings the original commenter has received under the post’s specified category. If the user comments on a different post under the same category, all of the ratings the user has received from past comments under the category will be used to calculate the trust rating of the user using the following formula.

Where n is the number of reviews the user has   written under said category and r is the average of all reviews the user has written under said category. Depending on a combination of the user’s trust rating and number of reviews written, they could receive a Verified badge for a particular category. The conditions for a user to receive the Verified badge under a category is as follows:

1. Number of reviews < 10, Trust Rating > 80
2. Number of reviews < 50, Trust Rating > 75
3. Number of reviews < 100, Trust Rating > 70

The Verified badge helps in identifying users that write consistently good reviews for a particular category. This way a user can be verified only for certain categories and not a generalized verified user.

Conclusion

This project is intended to help people looking for reliable sources to validate any piece of information. In terms of scope for the future, the findings in this project could be adopted by departments of government to filter out fabricated information and curb the spread of false news, while simultaneously helping increase the trustworthiness of legitimate news sources.

References

[1] R. Pinto, Y. O. de Lima, C. E. Barbosa and J. M. de Souza, \"Towards Fact-Checking through Crowdsourcing,\" 2019 IEEE 23rd International Conference on Computer Supported Cooperative Work in Design (CSCWD), 2019, pp. 494-499, doi: 10.1109/CSCWD.2019.8791903. [2] Michael Soprano, Kevin Roitero, David La Barbera, Davide Ceolin, Damiano Spina, Stefano Mizzaro, Gianluca Demartini,The many dimensions of truthfulness: Crowdsourcing misinformation assessments on a multidimensional scale Information Processing & Management, Volume 58, Issue 6, 2021, 102710, ISSN 0306-4573, https://doi.org/10.1016/j.ipm.2021.102710. [3] S. Sethi, Ricky. (2017). Crowdsourcing the Verification of Fake News and Alter+native Facts. 315-316. 10.1145/3078714.3078746. [4] Gaozhao, D. (2020). Flagging Fake News on Social Media: An Experimental Study of Media Consumers’ Identification of Fake News. Social Science Research Network. [5] Lamprou E, Antonopoulos N, Anomeritou I, Apostolou C. Characteristics of Fake News and Misinformation in Greece: The Rise of New Crowdsourcing-Based Journalistic Fact-Checking Models. Journalism and Media. 2021; 2(3):417-439. https://doi.org/10.3390/journalmedia2030025 [6] Young Ae Kim, Minh-Tam Le, H. W. Lauw, Ee-Peng Lim, Haifeng Liu and Jaideep Srivastava, \"Building a web of trust without explicit trust ratings,\" 2008 IEEE 24th International Conference on Data Engineering Workshop, 2008, pp. 531-536, doi: 10.1109/ICDEW.2008.4498374.

Copyright

Copyright © 2022 Sudhanshu Basu Roy, Mrs. Jeyasudha J. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.