wikipedia notability purges

[Yakk]

The early patents and papers related to google search engine technology can be read.

Basically, it generates relevance estimates for webpages based on the topology of links and the pattern of content in the pages.

Now, this doesn't mean that it checks the WWII vet's website against other sources and determines if it is consistent. Rather, it uses it's algorithm to find "good" web sites, and uses link and content patterns to take "good" web sites and spread the "good"ness to other websites.

like the number of hits that site receives

Google has very little, if any, knowledge of how many hits a given website receives.

What you wrote here has *nothing* to do with what I wrote above it. I was commenting on the statement that I quoted, about how, according to you, the reason that wiki articles tend to come up at the top of a google search is because they're so relevant. It's not, it's because they've paid to be there

Google does not use how much money a website pays them to determine if the website is valid. You can buy ads, but those place you in a different area of the page than the search results. Despite your accusation, Wikipedia does not show up high on the Google search page because it pays to be there.

...

The core of the google algorithm involves large matrix multiplication. To do "basic" googling (circa 1995), you create a billion by billion matrix of the internet. Each column/row pair is a specific website. Insert 1s whenever a website links to another website. Raise the ridiculously huge matrix to a reasonably high power (say, 10).

You now have "how many ways can you reach this website using 10 steps of links". This is a first-approximation attempt at determining how linked-to a website is.

Now, there is more than this going on. You have anti-google-bait technologies (aimed at detecting google-bombing). You have term-word specific relevance. You have "store the entire internet in memory", which lets it display the context of the words you searched for.

There are also index biasing strategies (changing which websites are walked by google spiders) which have an impact on the results of google.

The context of each link is also taken into account in ways I do not know enough about. But it means that if enough people say "the greatest evil on earth" and link to microsoft, searching for the "greatest evil on earth" will get you the microsoft homepage.

The goal of all of this work is to use the decision to link as a form of "vote of confidence" in a website. When I'm talking about, say, WWII, I'm more likely to link to a WWII wiki article than the website of a holocaust revisionist. That indicates, as one small bit in a sea of data, that wikipedia (the WWII website) is a decent place to find out information about WWII.

If Wikipedia sucked and I knew of other sources that where better, I would might link to those other sources instead of Wikipedia. Wikipedia being more known would stay at the top -- but as more people learned about the alternative source and it's quality (assuming they agree with me), the number of links pointed at the alternative source would climb.

And that is an example of a "I vaguely understand how Google page rank works" level of understanding. Notice that I made lots of caveats -- I don't know how many parts of the Google engine work.

[Indon]

Well, I feel it's the right of the Wiki community to exclude information they don't feel is important.

Just as I feel that it's my right to not give them money because they exclude information I may feel is important.

Those webcomic artists are being nice; I'm never going to bother to tell Wikipedia that I've decided not to donate for whatever reason.

[22/7]

Google does not use how much money a website pays them to determine if the website is valid. You can buy ads, but those place you in a different area of the page than the search results. Despite your accusation, Wikipedia does not show up high on the Google search page because it pays to be there.

Yeah, I was talking about the ads, and poorly worded this. The relevance and reliability comment were arguments from the previous argument and I should have separated those. My apologies.

[fjafjan]

My point is, a search engine does not search for relevance of the associated links or how reliable the information is.

Not by proxy, ie google isn't a giant fact checking machine, but since people WANT those things, a good search engine should try and find them.

[tiny]

So... why don't you make a Webcomic-Wiki?
Wiki is freeware. You just need a server and some people who like a certein webcomic enough to write about it.

[fjafjan]

it exist already, comixipedia or something like that it's called.

[Malice]

So... why don't you make a Webcomic-Wiki?
Wiki is freeware. You just need a server and some people who like a certein webcomic enough to write about it.

That's all well and good. But it's counter to the idea of Wikipedia as a comprehensive source of human knowledge. The answer to "Wikipedia is having a problem with webcomics" is not "Well, I guess webcomics will go somewhere else".

[22/7]

I guess one has to first answer the question of "to what depth should Wikipedia go?" Should there be a page in there for *everything* or not, and where's the cutoff?

[Maurog]

Ultimately, I want to be connected to a database of everything, with easy access to any and all knowledge in any degree of broadness or narrowness. Like, being able to say "Computer, display an explanation of the string theory in words that a 10-year old can understand"; "Give me a pie chart of cat owners vs dog owners in the world"; "Show me that issue of that comic with the kitty proximity graph"; "Draw me a graph for number of school murders vs. the graph for gun laws strictness in USA states between 1900 and 2030"; "Display all news concerning raptors on a world map by source of article"... etc etc.

If Wikipedia doesn't want to make steps in the direction of such a database, screw Wikipedia! It will eventually happen anyway, built by people with better future vision (probably Google), and Wikipedia won't be part of it.

Seriously, when I search for something, the last thing I want to see is a page with "oh, sorry, there was an article here, but we removed it because we think it wasn't notable." Did anyone ask you to think? How about you shut up and give me everything you got, and let *me* judge whether it's good or not?

[Belial]

Well, I feel it's the right of the Wiki community to exclude information they don't feel is important.

Oh yes, it's their right. That doesn't make it any less a dick move. Or foolish.

[zenten]

Any article that sits around too long without proper citations (and what that means exactly is a whole other debate) should be deleted. Anything with proper citations should be able to stay (obviously there are other rules for what the actual content should be). The topic of the article is irrelevant here, if there are proper secondary sources on XKCD or Dinosaur Comics, then those should be able to stay. If there is no actual information on a given act of genocide, then it should be deleted, at least until there are proper secondary sources.

[chaosspawn]

I can't defend the choice to blanketly remove webcomics, but notability does seem like a good criterion to me. Wikipedia "is a multilingual, web-based, encyclopedia project operated by the not-for-profit Wikimedia Foundation.". As such it's goal (as I interpret it) is to serve as a (free) repository for knowledge. Even presuming there's no technical or financial consideration I think that information there should be limited, notability being one of those forms. Webcomics, like anything else, should have some sort of critical mass before being include-able. A big band like say Green Day is notable, my brother's band (which has only played at school) is not. Bands have myspace pages, not wikipedia pages. I think that is appropriate, as each site has a different goal.

[Belial]

As such it's goal (as I interpret it) is to serve as a (free) repository for knowledge. Even presuming there's no technical or financial consideration I think that information there should be limited, notability being one of those forms.

Are you trying to say the second statement follows from the first? Because I don't see it.

[22/7]

I think he's missing a "however" between the two.

[tiny]

That's all well and good. But it's counter to the idea of Wikipedia as a comprehensive source of human knowledge. The answer to "Wikipedia is having a problem with webcomics" is not "Well, I guess webcomics will go somewhere else".

What's bad about decentralisation? Or specialisation? It's not as if they kill comic artists.
Why waste energy by complaining about 'The' Wiki when you could put it into new articles in The WebcomicWiki (thx fjafjan)?

[Maurog]

And when I want to know about hammers, I will go to The ToolWiki. And when I want to know about Schwarzenegger, I will go to The ActorWiki? All the entries on tools and actors in Wikipedia are therefore completely redundant, and can be removed? No, sorry, I don't see your point at all. Either you want to be a complete database of knowledge or not. If not, you will eventually step aside, and let the real complete database take its rightful place.

Look! A Quantum Physics Wiki! Let's remove all Wikipedia articles on quantum physics!

[Eleyras]

So, I just looked on Wikipedia and found an entire page devoted to my high school. There's nothing special about my high school. Nothing happens there, except that approx 1900ish students go there. Why is it more notable than webcomics?

[SecondTalon]

C) Because somebody's paying for that, and the more stupid pages that get put up about somebody's garage band, the more money it costs. It might be a small amount of money but it adds up over time.

The average large page is 100kb, which is ~10000 pages per GB. Newegg has a 500gb hd on sale for $100 right now, so Wikipedia could store 5 million such junk pages for $100. The last donation rally I saw brought in like a million and a half dollars, and according to wikipedia, they have about 2 million pages right now.
I think they can handle a few more junk pages, don't you?

I'm just going to say.. the storage is one of the cheapest, if not the cheapest part of running Wikipedia. Bandwidth is where the money goes.

As for the webcomics thing.. it really depends on the webcomic in question. I figure it'd need to be running for at least two years before it would even warrant a wikipedia page. I'm also assuming that the pages in question went through the 5 day process of deletion.

[zenten]

C) Because somebody's paying for that, and the more stupid pages that get put up about somebody's garage band, the more money it costs. It might be a small amount of money but it adds up over time.

The average large page is 100kb, which is ~10000 pages per GB. Newegg has a 500gb hd on sale for $100 right now, so Wikipedia could store 5 million such junk pages for $100. The last donation rally I saw brought in like a million and a half dollars, and according to wikipedia, they have about 2 million pages right now.
I think they can handle a few more junk pages, don't you?

I'm just going to say.. the storage is one of the cheapest, if not the cheapest part of running Wikipedia. Bandwidth is where the money goes.

As for the webcomics thing.. it really depends on the webcomic in question. I figure it'd need to be running for at least two years before it would even warrant a wikipedia page. I'm also assuming that the pages in question went through the 5 day process of deletion.

Well, that depends.

If Matt Groening were to start up a web comic I'm sure it would warrant a page pretty quickly.

[no-genius]

If Matt Groening were to start up a web comic I'm sure it would warrant a page pretty quickly.

Not really. Though I would expect it would me mentioned in the Matt Groening page. See, Matt Groening is already notable - he did this show called the Simpsons? Apparently lots of people like it. (Must've sprung up overnight).

[The Mighty Thesaurus]

Alongside those McDonald's.

[Indon]

Well, I feel it's the right of the Wiki community to exclude information they don't feel is important.

Oh yes, it's their right. That doesn't make it any less a dick move. Or foolish.

Well, my point was that by going against the wishes of their users, they risk drying up their source of revenue, being donations.

Though, on the other hand, Wikipedia shouldn't purely be based on Consentual Reality.

[cs24]

If Matt Groening were to start up a web comic I'm sure it would warrant a page pretty quickly.

http://en.wikipedia.org/wiki/Life_in_Hell

Might become a web comic...

[JoshuaZ]

And I've actually had a friend that had one of his works featured in a major gaming magazine, but I guess that doesn't count.

Did you mention that/cite the article?

Yes, I believe that there was a scan of the article listed as a source in it... But apparently that wasn't good enough.

That should have been enough if there was almost any other sourcing. I'm a Wikipedia admin, so if you PM me with details I'll take a look and may get the deletion reviewed.

[Maurog]

My main issue with Wikipedia notability policies remains the same - the attitude towards the customer.

A recent example - I was using Google to find stuff as usual, and one of the links it came up with was tagged as a "bad" site. That is, Google told me "dude, this site is marked as bad, it's like, full of malware and stuff, are you sure you wanna go there?" And then it gave me the options. I chose not to visit it, because I trust Google, but it gave me the bloody options! It didn't say "I think this site is bad, therefore I'm not letting you in, and there is nothing you can do about it."

So sure, slap some warnings on, tag stuff as non-notable, but denying content based on your ideas of notability is not my idea of good customer service.

[JoshuaZ]

Ok, speaking as a Wikipedia administrator (http://en.wikipedia.org/wiki/User:JoshuaZ ) and as someone who has engaged in many of the big bad, evil deletions (http://en.wikipedia.org/w/index.php?tit ... huaZ&page= ), I feel compelled to explain the basic idea behind notability. There are two issues which come into play: first Wikipedia cannot have any original research (http://en.wikipedia.org/wiki/WP:NOR ). If original research were allowed then we'd have no way of judging which original research was correct, which wasn't and how much weight conflicting research should be given. Therefore, everything in Wikipedia has to be from reliable sources (http://en.wikipedia.org/wiki/WP:RS ). Now, that means among other things that in order to write an article about almost any topic that is long than "X is a Y" we need reliable sources that talk about the subject. Hence, the notability policy- the need for non-trivial, independent reliable sources (essentially reliable secondary sources). (Note that notability is a term -of-art. Both some Wikipedians and non-Wikipedians seem to misunderstand this and think that notability means importance but this isn't the case. There might be very important things that don't meat Wikipedia's Notability standard and there might be very unimportant things that do).

Second, but less important, we need a minimal barrier of notability since otherwise things would be unnavigable. See for example, http://en.wikipedia.org/wiki/John_Cox_% ... guation%29 - we have at least 18 people named John Cox who are notable (although only a handful seem to have articles so far. Wikipedia is always a work in progress). Now, John Cox is a pretty common name. Imagine if we included in that list every person who wanted a Wikipedia article who was named John Cox. The vast majority of people who would be looking for one of the notable people (because there is at least a high correlation between notability and importance). I estimate that we delete probably a hundred articles a day about garage bands, or kids in school, or someone who wants to post their resume, etc. Furthermore, Wikipedia is hard to maintain, and the quality is at best uneven even without the notability guideline. Without it, we would likely be swamped and we'd have little chance to protect against problems like people including libel or privacy issues in articles. Notability keeps us at a sane level.

I'm not always happy with notability and it sometimes leads to results that I think are wrong. For example, we just had a variety of Warcraft characters deleted including Sylvanas Windrunner, who I at least thought should probably have an article. But we couldn't find any reliable sources (if you know of any or run across any, please feel free to let me know by all means). So in summary, notabilty isn't great, but it's necessary.

Nov 19 - edited for grammar and spelling

[Maurog]

I wrote a long response, but I deleted it. There is just no point arguing over this. Wikipedia isn't going to change, and eventually will doom itself. That's what I believe in, and it will take some radical changes to convince me otherwise.

[JoshuaZ]

I wrote a long response, but I deleted it. There is just no point arguing over this. Wikipedia isn't going to change, and eventually will doom itself. That's what I believe in, and it will take some radical changes to convince me otherwise.

You'd probably be surprised at how much I agree with that sentiment. There are serious structural problems and it could likely collapse under its own weight if they aren't dealt with. Nor am I convinced that the notability standard as it currently stands is the best possible standard, so by all means feel free to argue.

[22/7]

so by all means feel free to argue.

But, but, but... you agree!

[JoshuaZ]

so by all means feel free to argue.

But, but, but... you agree!

No, I might agree. I agree that there are serious structural problems. He and I may disagree what they are. And I'm always interested to here more views on that matter (Given enough eyeballs are bugs are shallow and all problems with Wikipedia can be corrected). And I'm not completely happy with the notability guideline, but I don't know what I'd do to replace it, so more input is always interesting.

[22/7]

Fair enough. I misinterpreted earlier.

[zenten]

I wrote a long response, but I deleted it. There is just no point arguing over this. Wikipedia isn't going to change, and eventually will doom itself. That's what I believe in, and it will take some radical changes to convince me otherwise.

You'd probably be surprised at how much I agree with that sentiment. There are serious structural problems and it could likely collapse under its own weight if they aren't dealt with. Nor am I convinced that the notability standard as it currently stands is the best possible standard, so by all means feel free to argue.

Well, it's less of a Wikipedia structural solution, and more of a userbased solution, but look at say World of Warcraft. World of Warcraft is notable enough to have it's own page. Many of the details of it are not warranted even in its own article however. So, if you have a bunch of information you want about World of Warcraft, create a wiki on the subject (or hijack another suitable one, like a complete Warcraft wiki), and link to the wiki from the Wikipedia page.. So if someone wants to learn more about World of Warcraft, they can just go there.

[Delbin]

A favorite author of mine wrote a much better argument than I could on the subject.

http://www.alexandraerin.com/?p=36

Here's the main point:

While Britannica and other venerable institutions of the encyclopedia business struggle with the question of “How do we remain relevant in this ever-changing world?”, Wikipedia’s asking the question, “How can I get to be where Britannica is now?” I’m reminded of a cartoon Kris Straub drew of a man motoring out to catch the sinking Titanic, as a metaphor for a cartoonist chasing a syndication deal.

Wikipedia has tremendous appeal to people who want to know about Pokemon species, X-Files episodes, webcomics, and so on. Its standards are perfectly up to snuff in areas such as those and the people looking for that kind of information are perfectly willing to trust an online resource. How does Wikipedia respond? By deleting episode guides and webcomic articles en masse, hoping that this will somehow please the critics and naysayers among the scientific and academic communities. It’s like they think that every webcomic article deleted improves the standing and quality of one scientific article.

The ultimate reason for excising these articles is because they’re “unencyclopedic”, which is another way of saying, “because Britannica doesn’t have such things!” Wikipedia’s determined to outdo its competitors by refusing to outdo them. Is that some weird kind of chivalry, or what?

[Belial]

....perfect. That quote pretty much exactly sums it up.

[bigglesworth]

Can you lock the thread now please? (And delete this post if you do so, obviously).

[Shadic]

...Why exactly should this topic be closed? Why would you want it to? As far as I can tell, it's an intelligent discussion, with interesting points brought up. (JoshuaZ, being the Wikipedia Editor that he is, does bring in an interesting viewpoint.)

However, I agree with the quote that Delbin posted entirely. Wikipedia's advantage is in it's huge amount of articles. If I see something and would like to find more information on it, my first bet it to go to Wikipedia. Hell, sometimes I've had to check the history of a non-existing article, in order to read a deleted copy.

Why? Because it wasn't notable. Besides the fact that I stumbled upon whatever subject it was, wanted more information, and decided to check the only source that I think would have a chance to have it, Wikipedians have decided to delete it. That is, in my opinion, stupidity.

Oh, and I hate the hierarchy system that's in place with that community. If somebody tags a bunch of articles for deletion based on notability, their opinion is worth more than somebody who happens to stumble upon the article, see that it's up for deletion, and would like to keep it. Shouldn't notability be based on if people can find it/care about it?

[JoshuaZ]

A favorite author of mine wrote a much better argument than I could on the subject.

http://www.alexandraerin.com/?p=36

Here's the main point:

While Britannica and other venerable institutions of the encyclopedia business struggle with the question of “How do we remain relevant in this ever-changing world?”, Wikipedia’s asking the question, “How can I get to be where Britannica is now?” I’m reminded of a cartoon Kris Straub drew of a man motoring out to catch the sinking Titanic, as a metaphor for a cartoonist chasing a syndication deal.

Wikipedia has tremendous appeal to people who want to know about Pokemon species, X-Files episodes, webcomics, and so on. Its standards are perfectly up to snuff in areas such as those and the people looking for that kind of information are perfectly willing to trust an online resource. How does Wikipedia respond? By deleting episode guides and webcomic articles en masse, hoping that this will somehow please the critics and naysayers among the scientific and academic communities. It’s like they think that every webcomic article deleted improves the standing and quality of one scientific article.

The ultimate reason for excising these articles is because they’re “unencyclopedic”, which is another way of saying, “because Britannica doesn’t have such things!” Wikipedia’s determined to outdo its competitors by refusing to outdo them. Is that some weird kind of chivalry, or what?

There's some truth to this. I don't know if it applies to notability issues by themselves, but there are areas where this does ring true. The obvious example that comes to my mind is the repeated arguments about whether or not Wikipedia should have spoiler tags (right now the answer is no). The primary argument for not having the tags is that other encyclopedias don't have such tags.