Metafilter Frequency Tables: 13 years, 636 million words

I’ve updated today a project I first launched a couple years ago: Metafilter Frequency Tables, a collection of tables calculated from the last 13 years of comments made by users on Metafilter.com.

The tables break down word frequency for the site as a whole as well as by several major subsites, and for all time as well as by year, month, and day. Each table includes raw count and parts-per-million data for each word. It’s all generated by some perl I wrote that fetches comments from our database and tabulates it into these text files. Read about the methodology here.

This isn’t by far the most carefully constructed set of such tables out there — I am a hobbyist, not a trained linguist, and this whole effort is very much DIY — but it’s the largest I’m aware of focused specifically on this sort of internet-mediated casual textual conversation over the last decade-plus, and I’m hoping it will be of some use or interest to word nerds.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>