Submissions/Article Feedback: Building Metrics from Reader Input


Submission no. 5005
Title of the submission
Article Feedback: Building Metrics from Reader Input
Type of submission (discussion, hot seat, panel, presentation, tutorial, workshop)
Presentation
Author of the submission
Adam Hyland
E-mail address
Special:EmailUser/Protonk
Username
Protonk
Country of origin
United States
Affiliation, if any (organisation, company etc.)
Personal homepage or blog
ShiftCommandAwesome
Abstract (at least 300 words to describe your proposal)
The Article Feedback Tool (AFT) was deployed on the English Wikipedia from 2010 to 2014 as a means to improve reader engagement, allowing reader to quickly provide input on articles. Anonymous and registered editors were given a chance to provide a quick numerical rating (for version 4) or an up/down assessment (for version 5) of utility on the article page itself. Although eventually discontinued as an engagement tool, AFT provides a valuable window into reader sentiment toward articles.
In 2012 I used a small dataset from the rollout of AFT version 4 to create a metric of reader sentiment. Using over 40 million ratings collected across a 3 year period, I build on that work by categorizing article subjects and recording article changes over time, incorporating content changes, subject features and peer review status.
In the aggregate, the feedback provides a window to content quality at a scale which is difficult to match with traditional surveys. Article feedback was offered over the life of an article, the content and (less commonly) the subject change over time. Most publicly available feedback data covers evolving sentiment toward a fixed artifact; IMDb raters may sour on Shrek, but it's the same movie. Content on Wikipedia is constantly changing and AFT data reflect this, allowing us to (partially) disambiguate opinions of content and subject--a common problem in measuring sentiment toward news or content on the web.
Even though the tool is no longer active, the data offers valuable insight into both Wikipedia articles as well as rating behavior in general. I will provide a brief tour of the data, model and results as well as potential jumping off points for other research. Finally, I offer suggestions on what, if anything, should replace AFT as a reader engagement or data gathering tool for editors and readers.
Track
Technology, Interface & Infrastructure
Length of session (if other than 30 minutes, specify how long)
30 minutes
Will you attend Wikimania if your submission is not accepted?
Yes
Slides or further information (optional)
Earlier submission is here and slides for same are here. I expect (given the different track and the more complete dataset and model) that the talk will be a bit more technical, spending less time on the various tranches of peer review and nature of the AFT system and more on the results and nature of the model. However, this isn't an academic talk, so the focus will remain on data visualization over tables and equations.
Special requests


Interested attendees

If you are interested in attending this session, please sign with your username below. This will help reviewers to decide which sessions are of high interest. Sign with a hash and four tildes. (# ~~~~).

  1. Add your username here.