Based and confused: Tracing the political connotations of a memetic phrase across the Web

Datasets for the case study on the spread of the vernacular term "based" across 4chan/pol/, Reddit, and Twitter. Data was gathered in November 2021. All files are anonymised as much as possible. They contain: Posts and tweets mentioning "based". False positives were filtered out until the precision was over 0.9. See filtering steps below. Twitter: Tweets mentioning "based" between 2010-01-01 and 2021-11-01. Collected via the Twitter v2 API. Retweets not allowed. Reddit: Comments mentioning "based" between 2010-01-01 and 2021-11-01. Collected via the Pushshift API. Data from deleted subreddits may be absent in their last month. 4chan/pol/: Posts and comments mentioning "based" between 2013-11-28 and 2021-11-01. Data derived from 4plebs and 4CAT. Counts per day and month for the datasets above. Sampled and annotated posts from three time slices. Includes the 200 most-liked or highest-scoring posts from Twitter and Reddit, and a random sample for 4chan/pol/. Annotated with general comments, who was deemed based, and whether the post concerns meta-discussion on the term. Time slices: July 2012 January 2017 September 2021 The most-common words between "based " and "pilled" on all three platforms. Table 2 below details the queries we carried out for the collection of the initial datasets. For all platforms, we chose to retain non-English languages since the diffusion of the term in other languages was also deemed relevant. Initial queries for the datasets source query query type Twitter (#based OR (based (pilled OR pill OR redpilled OR redpill OR chad OR virgin OR cringe OR cringy OR triggered OR trigger OR tbh OR lol OR lmao OR wtf OR swag OR nigga OR finna OR bitch OR rare) ) OR " is based" OR "that\'s based" OR "based as fuck" OR "based af" OR "too based" OR "fucking based" "extremely based" OR "totally based" OR "incredibly based" OR "very based" OR "so based" OR "pretty based" OR "quite based" OR "kinda based" OR "kind of based" OR "fairly based" OR "based ngl" OR "as based as" OR "thank you based " OR "stay based" OR "based god") -"based in"-"based off"-"based * off"-"based around"-"based * around"-"based on"-"based * on"-"based out of"-"based upon"-"based * upon"-"based at"-"based from"-"is based by"-"is based of"-"on which * is based"-"upon which * is based"-"which is based there"-"is based all over"-"based more on"-"plant based"-"text based"-"turn based"-"need based"-"evidence based"-"community based" -"web based" -is:retweet -is:nullcast Twitter v2 API Reddit based -"based in" -"based off" -"based around" -"based on" -"based them on" -"based it on" -"evidence based" Pushshift API 4chan/pol/ lower(body) LIKE '%based%' AND lower(body) NOT SIMILAR TO '%(-based|debased|based in |based off |based around |based on |based them on|based it on|based her on|based him on|based only on|based completely on|based solely on|based purely on|based entirely on|based not on |based not simply on|based entirely around|based out of|based upon |based at |is based by |is based of|on which it is based|on which this is based|which is based there|is based all over|which it is based|is based of |based firmly on|based off |based solely off|based more on|plant based|text based|turn based|need based|evidence based|community based|home based|internet based|web based|physics based)%' PostgreSQL There were some data gaps for 4chan/pol/ and Reddit. /pol/ data was missing because of gaps in the archives (mostly due to outages). The following time periods are incomplete or missing entirely: 15 - 16 April 2019 14 - 15 December 2019 3 - 10 December 2020 29 March 2021 10 - 12 April 2021 16 - 18 August 2021 11 October 2021 The 4plebs archive moreover only started in November 2013, meaning the first two years of /pol/’s existence are missing. The data returned by the Pushshift API did not return posts for certain dates. We somewhat mitigated this by also retrieving data through the new Beta endpoint. However, the following time periods were still missing data: 1 - 30 September 2017 1 February - 31 March 2018 5 - 6 November 2020 23 March 2021 through 27 March 2021 10 - 13 April 2021 Afterward initial data collection, we carried out several rounds of filtering to get rid of remaining false positives. For 4chan/pol/, we only needed to do this filtering once (attaining 0.95 precision), while for Twitter we carried out eight rounds (0.92 precision). For Reddit, we formulated nearly 500 exclusions but failed to generate a precision over 0.9. We thus had to do more rigorous filtering. We observed that longer comments were more likely to be false positives, so we removed all comments over 350 characters long. We settled on this number on the basis of our first sample; almost no true positives were over 350 characters long. Furthermore, we removed all comments except for those wherein <em>based</em> was used as a standalone word (thus excluding e.g. “plant-based”), at the start or end of a sentence, in capitals, or in conjunction with certain keywords or in certain phrases (e.g. “kinda based”). We also deleted posts by bot accounts by (rather crudely) removing posts of usernames including ‘bot’ or ‘auto’. This finally led to a precision of 0.9. Filters used for the 4chan/pol/ data. -based|location based Filters used for the Twitter data. @-mentions with “based” "on which &lt;max. 25 characters&gt; is based" "where &lt;max. 25 characters&gt; is based"<br> "kinda based &lt;max. 15 characters&gt; off " "wherever &lt;max. 15 characters&gt; is based" <br> #based #customer| alkaline based| anime based | are based near | astrology based | at the based of| b0Iuip5wnA| based economy| based game | based locally| based my name | based near | based not upon| based points| based purely off| based quite near | based solely off| based soy source| based upstairs| blast based| class based| clearly based of this| combat based| condition based| dos based| emotional based| eth based| fact based| gender based| he based his | he's based in | indian based| is based for fans| is based lies| is based near | is based not around | is based not on | is based once again on | is based there| is based within| issue based| jersey based| listen to 01 we rare| music based| oil based| on which it's based| page based 1000| paper based| park based | pc based| pic based| pill based regimen| puzzle based| sex based | she based her | she's based in | skill based| story based| they based their | they're based in| toronto based| trigger on a new yoga 2| u.s. based| universal press| us based| value based| we're based in | where you based?| you're based in |#alkaline #based|#apps #based|#based #acidic|#flash #based|#home #based|#miami #based|#piano #based|#value #based|american based|australia based|australian based|based my decision|based entirely around|based entirely on|based exactly on |based her announcement|based her decision|based her off|based him off|based his announcement|based his decision|based largely on|based less on|based mostly on|based my guess|based only around|based only on|based partly on|based partly upon|based purely on |based solely around|based solely on|based strictly on|based the announcement|based the decision|based their announcement|based their decision|based, not upon|battery based|behavior based|behaviour based|blockchain based|book based series|canon based|character based|cloud based|commision based|component based|computer based|confusion based|content based|depression based|dev based|dnd based|factually based|faith based|fear based|flash based|flintstones based|flour based|home based|homin based|i based my|interaction based|is based circa|is based competely on|is based entirely off|is based here|is based more on|is based outta|is based totally on |is based up here|is based way more on|live conferences with r3|living based of|london based|luck based|malex based|market based|miami based|needs based|nyc based|on which the film is based|opinion based|piano based|point based|potato based|premise is based|region based|religious based|science based|she is based there|slavery based show|softball based|thanks richard clark|u.k. based|uk based|vendor based|vodka based|volunteer based|water based|where he is based|where the disney film is based|where the military is based|who are based there|who is based there|wordpress cms Filters used for the Reddit data. Allowed all posts: “based” and one of the following keywords:<br> "left", "right", "lib", "trump", "fren", "god", "pill", "chad", "virgin", "cring", "trigger", "tbh", "lol", "lmao", "wtf", "damn", "swag", "nigga", "finna", "bitch", "rare", "department", "amogus" “based” in one of the following collocations:<br> "is based", "&gt;based ", "based and ", "the based ", "that is based", "that's based","this is based","he's based","she's based","they're based","so based","totally based","actually based","too based","very based","incredibly based","totally based","extremely based","fucking based","unfathomably based","based as fuck","based af","unironically based","the most based","super based","ultra based","utterly based","completely based","pretty based","quite based","kinda based","kind of based","fairly based","based ngl","as based as","stay based","keep based","ty based ","thank you based","thanks based","bless you based" Posts were based was used in a certain format:<br> BASED (all caps)<br> Based (capitalised)<br> based (at the start of a comment)<br> based (at the end of a comment)<br> Based.<br> \nBased<br> based.<br> . based<br> #based Excluded all posts with the following patterns: based AND system (e.g. “ARM based system) "based &lt;max. 25 characters&gt; on" "based &lt;max. 25 characters&gt; off" "based &lt;max. 25 characters&gt; around" "where &lt;max. 25 characters&gt; is based" "where &lt;max. 25 characters&gt;re based" "where &lt;max. 25 characters&gt; are you based" Excluded all posts with the following collocations: amd based| animal based| app based| arch based| are based inside| are based near| area based| ar

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.