Submissions/Article Feedback: Building Metrics from Reader Input

Submission no. 5005

Title of the submission: Article Feedback: Building Metrics from Reader Input

Type of submission (discussion, hot seat, panel, presentation, tutorial, workshop): Presentation

Author of the submission: Adam Hyland

E-mail address: Special:EmailUser/Protonk

Username: Protonk

Country of origin: United States

Affiliation, if any (organisation, company etc.)

Personal homepage or blog: ShiftCommandAwesome

Abstract (at least 300 words to describe your proposal)

The Article Feedback Tool (AFT) was deployed on the English Wikipedia from 2010 to 2014 as a means to improve reader engagement, allowing reader to quickly provide input on articles. Anonymous and registered editors were given a chance to provide a quick numerical rating (for version 4) or an up/down assessment (for version 5) of utility on the article page itself. Although eventually discontinued as an engagement tool, AFT provides a valuable window into reader sentiment toward articles.

In 2012 I used a small dataset from the rollout of AFT version 4 to create a metric of reader sentiment. Using over 40 million ratings collected across a 3 year period, I build on that work by categorizing article subjects and recording article changes over time, incorporating content changes, subject features and peer review status.

In the aggregate, the feedback provides a window to content quality at a scale which is difficult to match with traditional surveys. Article feedback was offered over the life of an article, the content and (less commonly) the subject change over time. Most publicly available feedback data covers evolving sentiment toward a fixed artifact; IMDb raters may sour on Shrek, but it's the same movie. Content on Wikipedia is constantly changing and AFT data reflect this, allowing us to (partially) disambiguate opinions of content and subject--a common problem in measuring sentiment toward news or content on the web.

Even though the tool is no longer active, the data offers valuable insight into both Wikipedia articles as well as rating behavior in general. I will provide a brief tour of the data, model and results as well as potential jumping off points for other research. Finally, I offer suggestions on what, if anything, should replace AFT as a reader engagement or data gathering tool for editors and readers.

Track: Technology, Interface & Infrastructure

Length of session (if other than 30 minutes, specify how long): 30 minutes

Will you attend Wikimania if your submission is not accepted?: Yes

Slides or further information (optional): Earlier submission is here and slides for same are here. I expect (given the different track and the more complete dataset and model) that the talk will be a bit more technical, spending less time on the various tranches of peer review and nature of the AFT system and more on the results and nature of the model. However, this isn't an academic talk, so the focus will remain on data visualization over tables and equations.

Special requests

Interested attendees

If you are interested in attending this session, please sign with your username below. This will help reviewers to decide which sessions are of high interest. Sign with a hash and four tildes. (# ~~~~).

Add your username here.