PLOS, Please publish our articles on Wednesdays: A look at altmetrics by day of publication

One of the most fun parts of doing quantitative research is the exploratory analysis that often precedes a more rigorous and focused attempt at answering a research question. At the early stages of a research project, plotting different variables with only a vague notion of a question in mind can help determine what the data “look like.” At this stage in the process, all explorations are equally valid. Unexpected relationships or patterns are uncovered without having to worry about statistical models or significance of relationships, or whether the uncovered pattern answers a “research question” (whatever that means!). One need not have an advanced knowledge of mathematics and statistics in order to look at and learn from data. Of course, such knowledge is useful for analyzing and understanding the data more fully, but the ability and knowledge required to extract, manipulate, and interpret data, indeed, can be developed by anyone with enough intellectual curiosity and desire to challenge their theoretical or heuristic assumptions. Metrics and measurement are a powerful strategic tool for understanding the world around us, and every student—whether a major in business, publishing or software engineering—should have an opportunity to familiarize themselves and experiment with it. This is why metrics & measurement feature in the seminar course Technology and Evolving Forms of Publishing, and why data analysis was a project option for the Technology Project course at SFU’s Master of Publishing Program. It is hoped that through these courses, the Master of Publishing students learn the value and limits of working with quantitative data. One such group of four students—Team Commander Data—decided they were up to the challenge. They chose to explore the PLOS Article Level Metrics (ALM) dataset. This particular version of the dataset included all metrics collected by the PLOS Lagotto application, for all PLOS articles published up until February 9, 2015. The team, however, only analyzed the articles published in 2014. Team Commander Data are not the first to use these data or other datasets like it, as the number of studies on social media metrics (altmetrics) continues to grow. In fact, earlier this month, a special issue of ASLIB Proceedings focused on social media metrics was published. Clearly, social media metrics are a current topic in need of more researchers asking critical questions that will have resounding implications for the scholarly community around the world, such as: "Will publishing an article on one day of the week lead to more social media mentions than on another?" Team Commander Data set out to answer this very important question, and the results were a little surprising. The team looked at three of the most widely used social media channels—Twitter, Facebook, and Mendeley (the academic social network/reference manager)—and it looks as though articles published closer to the middle of the week receive more mentions on Twitter and Facebook. This pattern holds regardless of whether one focuses on the median or the mean (although it probably makes more sense to look at the median, given the that the variables are not normally distributed). The box plot below shows the mean (the line that goes across the boxes), the median (the division between the light and dark grey), the first and third quartiles (top and bottom of boxes), and the first standard deviation (the “whiskers” on the boxes). Figure 1. Distribution of Altmetrics for PLOS Articles from 2014 by Weekday Table 1. Twitter tweets for PLOS Articles from 2014 by Weekday Monday Tuesday Wednesday Thursday Friday N 6,247 7,073 6,990 8,827 6,325 Median 35 66 79 57 35 Standard Deviation 163 292 320 249 201 Mean 64 114 133 99 77 Maximum 695 1,459 1,355 1,077 2,273 Table 2. Facebook posts for PLOS articles from 2014 by weekday Monday Tuesday Wednesday Thursday Friday N 6,247 7,073 6,990 8,827 6,325 Median 74 90 118 92 73 Standard Deviation 209 353 498 211 217 Mean 97 134 199 113 98 Maximum 1,013 1,884 2,575 822 1,149 Table 3. Mendeley saves for PLOS articles from 2014 by weekday Mendeley Monday Tuesday Wednesday Thursday Friday N 6,247 7,073 6,990 8,827 6,325 Median 21 33 31 39 23 Standard Deviation 42 109 64 93 48 Mean 24 46 37 50 27 Maximum 98 454 127 180 91 One possible explanation is that social media mentions happen very close to the date of publication—within a couple of days—and that people sharing research articles on social media are most active in the middle of the week. The dataset included no data on what time the mentions happened, but it was possible to explore the relationship between time and the metrics in a little more detail by looking at the average number of mentions per month (tweets, posts, or saves) for all articles published in 2014, to see how metrics evolve over time: Mendeley saves take a long time to accumulate, so older articles have much higher saves than newer ones; Twitter articles, however, must be happening close to the publication date, as there is no decrease over time; and Facebook is somewhere in between (but closer to Twitter’s pattern). Figure 2. Average Altmetrics for PLOS Articles from 2014 by Month This initial analysis maps onto our common-sense understanding of how people use Facebook, Twitter, and Mendeley. It also showcases the difficulty of doing any analysis that spans a significant time period. For meaningful results, all analyses must take into account that older articles have had more time to accumulate mentions than newer articles. One clever technique, the “Sign Test,” can be used for this purpose (see it in action in this paper). While it does not help us fully answer our initial question, the result is consistent with the assumption in our hypothesis that Facebook posts and Twitter mentions happen closer to the date of publication than Mendeley saves. Of course, more analysis is always needed; yet, as our research reminds us, any exploration, even the most seemingly frivolous, can yield unexpected results and raise interesting questions, thus enhancing our understanding of the world. Please leave us your comments with your own interpretations and ideas about how to take our findings further. Or, better yet, download the data and perform some analysis yourself! Data Citation ALM, PLOS (2015): Cumulative PLOS ALM Report - February 2015. figshare. http://dx.doi.org/10.6084/m9.figshare.1367535

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.