Wikipedia Articles Found To Be Too Hard To Read

Seen something interesting in the news or on the intertubes? Discuss it here.

Moderators: Zamfir, Hawknc, Moderators General, Prelates

Princess Marzipan
Posts: 7717
Joined: Sun May 27, 2007 5:28 am UTC
Location: neither a road, nor an island

Wikipedia Articles Found To Be Too Hard To Read

Postby Princess Marzipan » Fri May 24, 2013 3:37 am UTC

And ironically, the article itself uses some big words too.

Spoiler:
I’ve been excoriated a few times in comment threads when I’ve asserted that scientific information made accessible through open access (OA) publishing doesn’t necessarily give users access to the information — that actually, unless you’re well-educated and possess specialist knowledge, much of the information in the scientific literature will be inaccessible simply because you can’t understand it, much less put it to use.

With some experience teaching literacy to adults and English to non-English speakers, I know you can’t take language mastery for granted. Speaking isn’t the same as reading, fluency in daily activities doesn’t equate to comprehension of abstract or specialized information, and comprehension goes beyond words and sentences to realms and signals that are often “meta” to the words and sentences used.

Most scientific papers use a vocabulary pitched to a highly educated audience. It’s then sprinkled with jargon for a specialist audience familiar with statistics and trial design (quick, name the differences between a cohort, cross-over, or cluster trial design). Making sense of even the methods requires appreciable amounts of knowledge.

But let’s begin with readability.

A common measure of basic readability is the Flesch score. Standard reading level is at a Flesch score of about 60 or higher, with lower scores equating to more difficult reading. The scientific literature is pegged in various studies to have a reading score of around 30, with one 2002 BMJ study finding that British English is actually more readable than US English. A low readability score — lower equals more demanding — makes the information difficult or impossible to comprehend for the majority of non-scientific readers.

But how difficult a challenge is readability for Wikipedia readers? A study in 2010 of cancer information on Wikipedia found that while the information was as accurate as that in a professionally curated database, its advanced reading demands made it much less accessible (the professional database required about a 9th grade reading level, while Wikipedia’s cancer information required about a college sophomore’s reading skills).

Leveraging this and other information, a group of researchers in the Netherlands analyzed the readability of Wikipedia overall. Their paper, published in First Monday, is quite readable.

Wikimedia, the parent of Wikipedia, has known for some time that its core product is too difficult to read — to address this, it introduced a “Simple English” version of Wikipedia in 2003, which contains about 68,000 articles (compared to the 3.5 million in the main wiki). The Basic English suggested for the Simple English Wikipedia consists of about 850 words. In late 2003, the Simple English version had a readability score of 80 (Easy). By 2006, this score had dropped to 70 (Fairly Easy).

One interesting finding in this study is how many Wikipedia articles consist of five or fewer sentences. In filtering a few million articles, they found that 40% of Wikipedia articles consist of five sentences or less.

Eliminating these short articles, the authors found that the readability score of the majority of Wikipedia (73.5%) is 51.18 (SD = 13.84), which is lower than the goal of 60 (Standard). In addition, 45% of the articles could be qualified as Difficult or worse.

In the Simple English Wikipedia, 60% of the articles consist of five sentences or fewer. Eliminating these from the calculations, the reading score for the Simple English Wikipedia came in at 61.69, lower than the goal of 80. In fact, 94.7% of the articles scored lower than 80, and 42.3% were below the score needed to be considered Standard reading material.

The authors took a further step — comparing articles with the same title between the two versions. Comparing these, they found that the Simple English Wikipedia had an average score of 61.46 compared to a reading score of 49.27 for the general Wikipedia — both well off their reading level goals.

As the authors write in their conclusions:

The results of this study show that the readability of the English Wikipedia is overall well below a desired standard. . . . Moreover, half the articles can be classified as difficult or worse. This finding confirms our hypothesis that numerous articles on Wikipedia are too difficult to read for many people.

The authors also created a tool you can use to test the readability of any entry in Wikipedia.

There are two simple explanations for the declining readability scores in both the standard and Simple English versions of Wikipedia. First, it’s a skeuomorph of an encyclopedia, and encyclopedias are supposed to have formal, passive language and sophisticated vocabulary. That’s what we think it should be. The second explanation — robust borrowing from authoritative resources — also likely contributes to the works’ difficulties in reaching the optimal readability scores.

In fact, experts are notoriously bad at writing for non-expert audiences. Readability problems in medicine plague patient-education materials, for example. A 1989 study of the readability of smoking cessation materials found that “a serious disparity existed between the reading estimates of smoking education literature and the literacy skills of patients.” In short, many patients couldn’t read about how to quit. A 2001 study of patient-education materials aimed at parents and discussing pediatric topics found that the reading level was about four grades higher than the intended goal. A 2004 study of patient-education materials written for family medicine patients found a similar disparity. A 2008 study of orthopaedic patient education materials found that only 10% achieved the desired readability level. It seems that if you want to find an area where every medical specialty group is failing, you need look no further than the readability of their patient-education materials.

I once worked on a publication aimed at patients that went through academic editors as part of its approval process. It quickly became clear that the writers — who initially were writing at an appropriate level — were helpless to avoid writing for these academic editors after a time, because the feedback from these “experts” was firmly and unquestionably that too many nuances and qualifications were being left out by using short sentences and a limited vocabulary. Needless to say, the publication folded, having attained a reading level only a graduate student would understand.

Even when they try, it’s difficult for advanced readers and writers to write to a lower reading level. It feels condescending, like you’re “dumbing it down,” and also prevents you from using sentence structures and vocabulary that come to you naturally. It’s hard work, and it’s unnatural. But it’s also extremely important to do it right.

Wikipedia is free, yet it presents readability barriers to many of its users. This suggests that the challenge of making scientific information approachable — and therefore, accessible — is much deeper, much more difficult, and much more complex than simply removing reader funding.
"It's Saturday night. I've got no date, a two-liter of Shasta, and my all-Rush mixtape. Let's rock!"
"I am just about to be brilliant!"
General_Norris, on feminism, wrote:If you lose your six Pokémon, you lost.

User avatar
Magnanimous
Madmanananimous
Posts: 3478
Joined: Wed Feb 17, 2010 7:11 pm UTC
Location: Land of Hipsters and Rain (LOHAR)

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Magnanimous » Fri May 24, 2013 3:52 am UTC

This makes me want to actually start editing Wikipedia seriously, instead of just fixing little things. I've definitely had trouble reading some articles about math and science topics, where I only start understanding it after twenty minutes of skimming through related articles. Explain Like I'm Five material is sort of hard to find, but it always gets right to the point in ways anyone can understand.

I wonder if the Simple English could be made a bigger part of the picture... There could be something like a colored toggle button at the top of the page that you can switch between green (intuitive understanding for unskilled audiences) and red (research for people who already understand the basics). Although the Simple wiki does only have 100,000 articles right now compared to 4.2 million.

User avatar
Izawwlgood
WINNING
Posts: 18638
Joined: Mon Nov 19, 2007 3:55 pm UTC
Location: There may be lovelier lovelies...

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Izawwlgood » Fri May 24, 2013 4:50 am UTC

I'm all for a separate collection of reference material aimed at the layman, but I'm not sure Wikipedia is that, or should be that. If I'm looking for something on Wikipedia, I don't want the 'explain it to me like I'm 5' material.
... with gigantic melancholies and gigantic mirth, to tread the jeweled thrones of the Earth under his sandalled feet.

User avatar
Woopate
Scrapple
Posts: 503
Joined: Fri Mar 12, 2010 10:34 am UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Woopate » Fri May 24, 2013 5:20 am UTC

Well it depends on your purpose there. If you are doing research for a project, technical terms may help you. If you are looking to get a beginner's grasp of the topic, it's less useful to go to a page that may as well be written in another language. I avoid wikipedia until I feel I have a decent understanding of the building blocks of that topic, so that I can understand what it's trying to say. I think that overall anything that restricts usage to a "free" service (and language barriers are definitely restrictive) is a problem. I don't thinknit should change the articles standard, but I agree making the simple English version a more easily accessible feature would be a good place to start.

User avatar
Adacore
Posts: 2755
Joined: Fri Feb 20, 2009 12:35 pm UTC
Location: 한국 창원

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Adacore » Fri May 24, 2013 5:26 am UTC

In terms of purely the level of English, since my reading level is pretty high, I prefer articles that are written at a fairly high level. I'm not talking about flat-out bad writing here, of which Wikipedia certainly has plenty*, but just vocabulary and, to a lesser extent, grammar choices. I understand why lots of government communications and things read like they think I'm in elementary school, but it still feels really condescending to me. Surely partly the answer to this is to try and ensure that when everyone leaves school they are able to read at a twelfth grade level? That would mean that a Flesch readability score of 40-50 would be fine for everyday writing aimed at adults.

Having said that, I do find that if I'm trying to jump into some complex area of science or maths I know very little about, it takes a long time and a lot of 'related article' reading to get up to speed. I'm not sure this is entirely undesirable though - if I want to understand a complex principle, isn't it a good thing that I have to learn about the simpler building blocks first?

*I did a little bit of Wikipedia copyediting a few years ago. It did feel like I was making a difference, but there was just such a mountain of requests for editing, piling up way faster than Wikipedia's meagre supply of copyeditors could handle them, that I just gave up. And I'm sure any articles I edited ended up with similar Flesch scores to those they had before - I just ensured they were better structured and more clearly phrased.

Incidentally, according to MS Word, the Flesch readability score of this article is about 28 (the first two paragraphs score 14, which gives the immediate impression that the article is very high level, but goes on to use shorter sentences after that, which brings the score up a bit). My normal writing style seems to score something around 40, varying a bit depending how crazy I go with commas and endless sentences.

User avatar
Magnanimous
Madmanananimous
Posts: 3478
Joined: Wed Feb 17, 2010 7:11 pm UTC
Location: Land of Hipsters and Rain (LOHAR)

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Magnanimous » Fri May 24, 2013 8:47 am UTC

The actual Flesch formula is 206.835 - 1.015*average sentence length - 84.6*average syllable length... Which seems a little arbitrary but okay. Word choice isn't a factor, although I'd guess that correlates strongly with the average number of syllables. (Also, from its Wiki article: "One particularly long sentence about sharks in chapter 64 of Moby-Dick has a score of - 146.77.")
Izawwlgood wrote:I'm all for a separate section of reference material aimed at the layman, but I'm not sure Wikipedia is that, or should be that. If I'm looking for something on Wikipedia, I don't want the 'explain it to me like I'm 5' material.

I'm not sure if it would be easier to expand the current Simple English material or just start a different project altogether. It probably makes more sense to have a separate wiki with the same framework and templates and everything, since the only real interaction between wikis would be a "see this on the other site" button somewhere, if that.

HungryHobo
Posts: 1708
Joined: Wed Oct 20, 2010 9:01 am UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby HungryHobo » Fri May 24, 2013 9:54 am UTC

actually, a metric that simple is good. it probably wouldn't be hard to write some sort of plugin for editors of simple.wikipedia to highlight sections which are likely to be a problem.
Give a man a fish, he owes you one fish. Teach a man to fish, you give up your monopoly on fisheries.

nitePhyyre
Posts: 1280
Joined: Mon Jul 27, 2009 10:31 am UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby nitePhyyre » Fri May 24, 2013 8:55 pm UTC

I’ve been excoriated a few times in comment threads ...
Pretty sure the author was aiming for irony with that.

Adacore wrote:Having said that, I do find that if I'm trying to jump into some complex area of science or maths I know very little about, it takes a long time and a lot of 'related article' reading to get up to speed. I'm not sure this is entirely undesirable though - if I want to understand a complex principle, isn't it a good thing that I have to learn about the simpler building blocks first?
Indeed. I imagine that this is the desired method of learning from an encyclopedia. If you want to sit down, read from the start to end, and understand something, grab a textbook.

Am I the only person who thinks that the problem isn't that the difficulty of reading material is too high, but that the reading ability of people is too low?
sourmìlk wrote:Monopolies are not when a single company controls the market for a single product.

You don't become great by trying to be great. You become great by wanting to do something, and then doing it so hard you become great in the process.

User avatar
Izawwlgood
WINNING
Posts: 18638
Joined: Mon Nov 19, 2007 3:55 pm UTC
Location: There may be lovelier lovelies...

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Izawwlgood » Fri May 24, 2013 9:11 pm UTC

nitePhyyre wrote:Am I the only person who thinks that the problem isn't that the difficulty of reading material is too high, but that the reading ability of people is too low?
Basically this; there are plenty of websites dedicated to teaching things. Wikipedia is an online reference, and should serve as a concise repository for information.
... with gigantic melancholies and gigantic mirth, to tread the jeweled thrones of the Earth under his sandalled feet.

User avatar
Magnanimous
Madmanananimous
Posts: 3478
Joined: Wed Feb 17, 2010 7:11 pm UTC
Location: Land of Hipsters and Rain (LOHAR)

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Magnanimous » Fri May 24, 2013 11:04 pm UTC

Izawwlgood wrote:Basically this; there are plenty of websites dedicated to teaching things. Wikipedia is an online reference, and should serve as a concise repository for information.
There are certainly a lot of educational sites, but most of the ones I've seen are less of a reference and more based on classes and interactivity. Which is fantastic and a great way to learn something, but I don't think access to concise reference material on basically everything should be limited to those with above-average reading skills. Probably 90% of the time I use Wikipedia is spent reading about random crap that I'm curious about, with the rest doing research for uni projects.

I could find the information elsewhere, but the benefit of the wiki is that virtually anything I'd want to know is on there in a simple and consistent format. I have the Wikipedia app on my phone and it feels good to know that at any given moment I'm twenty seconds away from knowing what something is. Yesterday, I noticed that my toothpaste prevents periodontitis. I had no idea what that is, and now I do. Most educational sites I've been to are specific to certain topics or don't go completely in-depth, or they're harder to navigate.

... But obviously that's just my experience. As long as everyone has free access to information, I wouldn't care about the format.

snow5379
Posts: 247
Joined: Tue Apr 12, 2011 6:06 pm UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby snow5379 » Sat May 25, 2013 3:43 am UTC

Wikipedia is meant to hold information... not teach it. Would an article on geometry have to first teach the reader algebra so they can understand it? That's ridiculous.

User avatar
addams
Posts: 9417
Joined: Sun Sep 12, 2010 4:44 am UTC
Location: Gold Beach, OR; 97444

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby addams » Sat May 25, 2013 3:56 am UTC

Am I the only person who thinks that the problem isn't that the difficulty of reading material is too high, but that the reading ability of people is too low?

No.

It seems that like University Educations and Old Poetry, Reading is not everyone's cup of tea.
It would be such a loss if Wiki became dumbed down so that people that don't want it can read it.
And; People that do want it no longer find it engaging.

Simple English Wiki as a support for the young and other learners seems to be The Right answer.
It may be fun for The Whole Family. Expert readers can read it.

Simple explanations are nice. But; To have real fun Wiki is Great.
It does not know everything. It knows a lot.
Life is, just, an exchange of electrons; It is up to us to give it meaning.

We are all in The Gutter.
Some of us see The Gutter.
Some of us see The Stars.
by mr. Oscar Wilde.

Those that want to Know; Know.
Those that do not Know; Don't tell them.
They do terrible things to people that Tell Them.

User avatar
Magnanimous
Madmanananimous
Posts: 3478
Joined: Wed Feb 17, 2010 7:11 pm UTC
Location: Land of Hipsters and Rain (LOHAR)

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Magnanimous » Sat May 25, 2013 8:45 pm UTC

snow5379 wrote:Wikipedia is meant to hold information... not teach it. Would an article on geometry have to first teach the reader algebra so they can understand it? That's ridiculous.
That is very ridiculous, and simple wiki pages should also make some assumptions about readers' prior knowledge. Though it's usually not too intrusive to briefly explain a related concept: on some articles about diseases, the paragraphs about etiology clarify that etiology is the study of what causes diseases.

Again, I don't want to make sweeping changes to the English Wikipedia (other than rewriting some sections, because people can be pretty bad writers). A separate wiki for layman explanations doesn't have to affect anything.

User avatar
Iulus Cofield
WINNING
Posts: 2917
Joined: Wed Apr 07, 2010 9:31 am UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Iulus Cofield » Sat May 25, 2013 10:26 pm UTC

A lot of broad topics on Wikipedia already are linked through portal pages/sidebars, it wouldn't be inconceivable to make portal pages also serve as introductions for laypeople.

johnny_7713
Posts: 555
Joined: Tue Jun 15, 2010 1:31 pm UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby johnny_7713 » Sat May 25, 2013 10:37 pm UTC

snow5379 wrote:Wikipedia is meant to hold information... not teach it. Would an article on geometry have to first teach the reader algebra so they can understand it? That's ridiculous.

That's not really what the article is complaining about. The article is complaining about the high Flesch score of average wikipedia articles, which is only about sentence and word length, not about how much prior knowledge is required to understand the text.

User avatar
Magnanimous
Madmanananimous
Posts: 3478
Joined: Wed Feb 17, 2010 7:11 pm UTC
Location: Land of Hipsters and Rain (LOHAR)

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Magnanimous » Sat May 25, 2013 11:11 pm UTC

johnny_7713 wrote:That's not really what the article is complaining about. The article is complaining about the high Flesch score of average wikipedia articles, which is only about sentence and word length, not about how much prior knowledge is required to understand the text.

Most articles are pretty readable for me, but I still find some that are just needlessly verbose. As a random example: U.S. State, which seems like a fairly simple concept. The opening has a score of 41.8, about a high school senior reading level. I'm betting the average high schooler could understand it after a few readthroughs, but that they'd treat it like legalese.
Spoiler:
A state of the United States of America is one of the fifty constituent political entities that shares its sovereignty with the United States federal government. Because of the shared sovereignty between each U.S. state and the U.S. federal government, an American is a citizen of both the federal republic and of his or her state of domicile.[1] State citizenship and residency are flexible and no government approval is required to move between states, except for persons covered by certain types of court orders (e.g., paroled convicts and children of divorced spouses who are sharing custody).

States are divided into counties or county-equivalents, which may be assigned some local governmental authority but are not sovereign. County or county-equivalent structure varies widely by state. Kentucky, Massachusetts, Pennsylvania, and Virginia use the official title of Commonwealth rather than State.

The United States Constitution allocates certain powers to the federal government. It also places limitations on the federal and state governments. State governments are allocated power by the people (of each respective state) through their individual constitutions. By ratifying the United States Constitution, the states transferred certain limited sovereign powers to the federal government. Under the Tenth Amendment, "all powers not delegated to the federal government nor prohibited to the states are retained by the states or the people."

Historically, the tasks of public safety (in the sense of controlling crime), public education, public health, transportation, and infrastructure have generally been considered primarily state responsibilities, although all of these now have significant federal funding and regulation as well (based largely upon the Commerce Clause, the Taxing and Spending Clause, and the Necessary and Proper Clause of the U.S. Constitution).

Over time, the U.S. Constitution has been amended, and the interpretation and application of its provisions have changed. The general tendency has been toward centralization and incorporation, with the federal government playing a much larger role than it once did. There is a continuing debate over states' rights, which concerns the extent and nature of the states' powers and sovereignty in relation to the federal government as well as the rights of individual persons. Debates over states' rights were a contributing factor to the outbreak of the American Civil War.

The United States Congress may admit new states on an equal footing with existing ones; this last happened in 1959 with the admission of Alaska and Hawaii. The U.S. Constitution is silent on the question of whether states have the power to leave unilaterally, or secede from, the Union, but the U.S. Supreme Court has ruled[2][3] secession to be unconstitutional, a position driven in part by the outcome of the American Civil War.

User avatar
Steax
SecondTalon's Goon Squad
Posts: 3038
Joined: Sat Jan 12, 2008 12:18 pm UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Steax » Sun May 26, 2013 2:05 am UTC

Could this also be an indicator that the Flesch-Kincaid test needs updating? I'm not sure if word/sentence length is a big deal on Wikipedia, where word complexity has a great tendency to be an issue. Sure, it's wikipedia, so intralinking is a thing, but we still can't have all blue articles, can we?
In Minecraft, I use the username Rirez.

Tirian
Posts: 1891
Joined: Fri Feb 15, 2008 6:03 pm UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Tirian » Sun May 26, 2013 2:07 am UTC

Someone should invent something like Wikipedia except that it would be researched, written, and edited by trained professionals instead of amateurs who are only paid with e-peen.

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 5496
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Thesh » Sun May 26, 2013 2:09 am UTC

Like some sort of paid encyclopedia?
Honesty replaced by greed, they gave us the reason to fight and bleed
They try to torch our faith and hope, spit at our presence and detest our goals

User avatar
Steax
SecondTalon's Goon Squad
Posts: 3038
Joined: Sat Jan 12, 2008 12:18 pm UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Steax » Sun May 26, 2013 2:16 am UTC

Like, hmm, paid for by Microsoft?
In Minecraft, I use the username Rirez.

Tirian
Posts: 1891
Joined: Fri Feb 15, 2008 6:03 pm UTC

Re: Wikipedia Articles Found To Be Too Hard To Read

Postby Tirian » Sun May 26, 2013 2:29 am UTC

Thesh wrote:Like some sort of paid encyclopedia?


Whoa, watch it with the big words there, Brainiac!

Steax wrote:Could this also be an indicator that the Flesch-Kincaid test needs updating?


Like any test, Flesch-Kincaid is not responsible for people who misuse the results. I'm all for government documents and insurance policies and wiki articles being comprehensible. But sentence length and sesquipedalian vocabulary are not the primary causes of unclear writing, and so using a formula based only on those two factors to "improve" readability is going to be gravely (though probably amusingly) disappointing.


Return to “News & Articles”

Who is online

Users browsing this forum: No registered users and 17 guests