Trout
August 16th 2004, 12:00 AM
Navigating the Information Jungle
A Brief Guide to Using Sources Critically
by James Patrick Holding
500
When I worked as a librarian I was often asked to recommend a "good source" for requested types of information. Up until now I have answered the same question from people writing me about Tekton, but have never systematized that advice into an article. It seems like a good idea to do that!
The questions I get are of the nature, "How do you know a source is reliable?" "Who do you trust and why?" "How do we decide between authorities -- especially when they are both well-credentialed?" The answer many people give is, "You can't. Let's throw up our hands and go watch the Super Bowl." The anwser others give is, "Just pick who you like best." The answer I give is: You follow some basic research guidelines. This is the sort of stuff librarians learn in school -- no, not just the Dewey Decimal system, or how to make people shut up in the library (we have nuclear bombs we use for that purpose); how to select sources and decide which ones are reliable -- and how to decide between them.
In this service I have some basic guidelines which can make things easier. Understand that none of these is a 100%, ironclad absolute -- there can be obvious exceptions; but the fewer of these expectations are met, the less likely it is you have something worth using in your hands.
1. Check their credentials! I've beaten this one into the ground that you probably guessed it, but I'll beat it some more and some more. Most sources will have some kind of short biography giving a person's credentials. It should tell you what their education has been, give a highlight of publishing credits (you can look for someone's vitae if you want a full list; with many scholars, they won't fit on a book's cover or flyleaf), and maybe name some prominent associations. If these are either:
1. Missing (the publisher may not have had room, or if your book is missing a paper cover, it may have been there -- but assuming otherwise...)
2. Not relevant (i.e., a book on Biblical scholarship written by someone with a Ph. D. in aerodynamics!)
3. Unclear (i.e., it says they "got a Ph. D. at Vanderbilt" but does not say in what)
....you'll need to think about that source a bit. Obviously such credentials are not required to be an author in the right (especially if one uses sources of such credit to compose a book) but they can help you decide whether an author is likely to have credibility.
2. Check their sources. A respectable source should have some sort of bibliography and/or notes. Look at these carefully. Warning signs on this are:
1. If you see the same sources used over and over, in other words, minimal variety in sources. This could indicate someone who hasn't done a lot of work themselves and is just copying one author uncritically (good example: Michael Martin using Wells in his "did Jesus exist" chapter, or Laurence Gardner using so much Barbara Theiring). Of course you'll need to consider other possibilities (is the author offering a refutation of or "update" to the source he uses?), but re-use of the same source repeatedly is often a danger sign.
2. If you see incomplete citatations or none at all -- if the source has no notes (just a bibliography), or bad ones (like The Hiram Key noting with just a title and author -- no page number!) there is likely malfeasance and/or incompetence at work. Though note that this is something that may work on a sliding scale: You don't need a note to the claim, "The sky is blue"; higher up, scholars will not need a note for something that is a "given" in their profession or their circle.
3. If you see sources that are overwhelmingly old -- unless the book is a review of history or something like that, high use of sources that aren't recent could be a sign of someone not doing critical research.
4. A veteran reader pointed to a related caution, one not so easy to detect -- someone who appeals to the same sources, AND to old ones, may be playing a game that does the same thing in essence! Their newer sources may just be rehashes of the old ones, and may themselves be using the same source! Also be cautious of people who cite a source, then "piggyback" their own statements onto what the source says. An example comes from a reader, who saw a dissertation in which the author mentioned the Inquisition. He used a credentialed source to back up a certain claim. In the next sentence, which was made seem to be a direct continuation to what was said earlier, the author gave a standard claim about the Inquisition claiming over 500,000 victims!
5. Finally, please note that "padding" -- listing sources in a bibliography that are not actually used, or are used sparingly -- is not uncommon. The recent great example of how Freke and Gandy made use of that "Dionysus on the cross" picture, while failing to report that one of their sources said it was declared a forgery by experts, speaks for itself!
3. Any recommendations? These days a good source may have recommendations from other writers or from review publications on their cover. These can be good to check. If good comments come from a fairly broad ideological spectrum (as is usually the case with Ben Witherington's books) you're likely in good shape. Recommends from a narrow spectrum (i.e., John Shelby Spong being recommended by John Dominic Crossan!) are less good worthwhile, but still good. Recommends from nobodies or relative nobodies in context (i.e., a movie star recommending Spong!!) don't help a bit. Prince Michael of Albant recommending Laurence Gardner is so ghastly it requires euthanasia. Recommends from professional reviewing sources like Publishers Weekly may be useful depending on the credentials of the reviewer or their experience. Watch out though, when the recommend seems too short, like Eisenman's use of Kirkus Review calling his book "fascinating reading" and nothing else (when the rest of the review was quite negative).
4. Reviews. If you want to dig deep, check for reviews of the book in peer-reviewed publications or in general reviewing sources like Kirkus. The former will usually be of more use than the latter since they will be by peers.
5. And what's not a good reason to prefer a source? It's sad to have to say this, but since people do think these are reasons a source is credible, they have to be mentioned:
1. "It's biased towards a point of view." Every source is "biased" to a point of view, and the truth is always biased. This is just an excuse for not doing critical homework. I don't throw out a book just because it comes from Prometheus Press, and I don't expect Skeptics to throw out a book just because it came from Word Books. Which relates to:
2. "It's published by a fundamentalist/atheist/cultic press." A caveat here is that this sometimes gives critical information about a writer, on certain levels. Acharya S obviously gives us a warning being published with "Steamshovel Press" (which also does stuff on UFOs and Atlantis!). This is a reliable indicator at extremes, but not so much in the middle.
3. "People on Amazon like it." Most Amazon reviews are done by everyday people and are a symptom of one of the illnesses of American pseudo-democracy. Be just as critical with them (if not more so) than anything else.
4. "It's a best seller!" So was Mein Kampf in pre-Nazi Germany. What's your point? By that logic you need to believe everything in the Bible at once.
5. "It's provocative/it opens your mind!" No comment.
Once again, these are just some general guidelines; they are far from ironclad, and they are certainly no substitute for logical and critical thinking. In fact, you really can't use one without the other. Have fun!
www.tektonics.org
500
Notice - The featuring of a particular article does not constitute endorsement of every single item or point of view contained therein by each and every member of TheologyWeb leadership. We strive to have a varied cross-section of representations of differing opinions on secondary Christian issues. The only requirement for the featuring of a particular article is that said article must not contradict the essentials articulated in the TheologyWeb statement of faith found here in our Mission Statement (http://www.theologyweb.com/forum/mission/)or be blatantly offensive to the Christian worldview of the site Owners.
Notice – The ministries featured in this section are guests of this site and very often not active members of debate forums. Additionally, this area is frequented and highlighted for guests who also very often are not acclimated to debate. As such, the rules of conduct here will be more strict than in the general forum. This will be something within the discretion of the Moderators, but we simply ask that you conduct yourselves in a manner considerate of the fact that these ministries are our invited guests. You can always feel free to start a related thread in general forum without such extra restrictions. Thank you.
A Brief Guide to Using Sources Critically
by James Patrick Holding
500
When I worked as a librarian I was often asked to recommend a "good source" for requested types of information. Up until now I have answered the same question from people writing me about Tekton, but have never systematized that advice into an article. It seems like a good idea to do that!
The questions I get are of the nature, "How do you know a source is reliable?" "Who do you trust and why?" "How do we decide between authorities -- especially when they are both well-credentialed?" The answer many people give is, "You can't. Let's throw up our hands and go watch the Super Bowl." The anwser others give is, "Just pick who you like best." The answer I give is: You follow some basic research guidelines. This is the sort of stuff librarians learn in school -- no, not just the Dewey Decimal system, or how to make people shut up in the library (we have nuclear bombs we use for that purpose); how to select sources and decide which ones are reliable -- and how to decide between them.
In this service I have some basic guidelines which can make things easier. Understand that none of these is a 100%, ironclad absolute -- there can be obvious exceptions; but the fewer of these expectations are met, the less likely it is you have something worth using in your hands.
1. Check their credentials! I've beaten this one into the ground that you probably guessed it, but I'll beat it some more and some more. Most sources will have some kind of short biography giving a person's credentials. It should tell you what their education has been, give a highlight of publishing credits (you can look for someone's vitae if you want a full list; with many scholars, they won't fit on a book's cover or flyleaf), and maybe name some prominent associations. If these are either:
1. Missing (the publisher may not have had room, or if your book is missing a paper cover, it may have been there -- but assuming otherwise...)
2. Not relevant (i.e., a book on Biblical scholarship written by someone with a Ph. D. in aerodynamics!)
3. Unclear (i.e., it says they "got a Ph. D. at Vanderbilt" but does not say in what)
....you'll need to think about that source a bit. Obviously such credentials are not required to be an author in the right (especially if one uses sources of such credit to compose a book) but they can help you decide whether an author is likely to have credibility.
2. Check their sources. A respectable source should have some sort of bibliography and/or notes. Look at these carefully. Warning signs on this are:
1. If you see the same sources used over and over, in other words, minimal variety in sources. This could indicate someone who hasn't done a lot of work themselves and is just copying one author uncritically (good example: Michael Martin using Wells in his "did Jesus exist" chapter, or Laurence Gardner using so much Barbara Theiring). Of course you'll need to consider other possibilities (is the author offering a refutation of or "update" to the source he uses?), but re-use of the same source repeatedly is often a danger sign.
2. If you see incomplete citatations or none at all -- if the source has no notes (just a bibliography), or bad ones (like The Hiram Key noting with just a title and author -- no page number!) there is likely malfeasance and/or incompetence at work. Though note that this is something that may work on a sliding scale: You don't need a note to the claim, "The sky is blue"; higher up, scholars will not need a note for something that is a "given" in their profession or their circle.
3. If you see sources that are overwhelmingly old -- unless the book is a review of history or something like that, high use of sources that aren't recent could be a sign of someone not doing critical research.
4. A veteran reader pointed to a related caution, one not so easy to detect -- someone who appeals to the same sources, AND to old ones, may be playing a game that does the same thing in essence! Their newer sources may just be rehashes of the old ones, and may themselves be using the same source! Also be cautious of people who cite a source, then "piggyback" their own statements onto what the source says. An example comes from a reader, who saw a dissertation in which the author mentioned the Inquisition. He used a credentialed source to back up a certain claim. In the next sentence, which was made seem to be a direct continuation to what was said earlier, the author gave a standard claim about the Inquisition claiming over 500,000 victims!
5. Finally, please note that "padding" -- listing sources in a bibliography that are not actually used, or are used sparingly -- is not uncommon. The recent great example of how Freke and Gandy made use of that "Dionysus on the cross" picture, while failing to report that one of their sources said it was declared a forgery by experts, speaks for itself!
3. Any recommendations? These days a good source may have recommendations from other writers or from review publications on their cover. These can be good to check. If good comments come from a fairly broad ideological spectrum (as is usually the case with Ben Witherington's books) you're likely in good shape. Recommends from a narrow spectrum (i.e., John Shelby Spong being recommended by John Dominic Crossan!) are less good worthwhile, but still good. Recommends from nobodies or relative nobodies in context (i.e., a movie star recommending Spong!!) don't help a bit. Prince Michael of Albant recommending Laurence Gardner is so ghastly it requires euthanasia. Recommends from professional reviewing sources like Publishers Weekly may be useful depending on the credentials of the reviewer or their experience. Watch out though, when the recommend seems too short, like Eisenman's use of Kirkus Review calling his book "fascinating reading" and nothing else (when the rest of the review was quite negative).
4. Reviews. If you want to dig deep, check for reviews of the book in peer-reviewed publications or in general reviewing sources like Kirkus. The former will usually be of more use than the latter since they will be by peers.
5. And what's not a good reason to prefer a source? It's sad to have to say this, but since people do think these are reasons a source is credible, they have to be mentioned:
1. "It's biased towards a point of view." Every source is "biased" to a point of view, and the truth is always biased. This is just an excuse for not doing critical homework. I don't throw out a book just because it comes from Prometheus Press, and I don't expect Skeptics to throw out a book just because it came from Word Books. Which relates to:
2. "It's published by a fundamentalist/atheist/cultic press." A caveat here is that this sometimes gives critical information about a writer, on certain levels. Acharya S obviously gives us a warning being published with "Steamshovel Press" (which also does stuff on UFOs and Atlantis!). This is a reliable indicator at extremes, but not so much in the middle.
3. "People on Amazon like it." Most Amazon reviews are done by everyday people and are a symptom of one of the illnesses of American pseudo-democracy. Be just as critical with them (if not more so) than anything else.
4. "It's a best seller!" So was Mein Kampf in pre-Nazi Germany. What's your point? By that logic you need to believe everything in the Bible at once.
5. "It's provocative/it opens your mind!" No comment.
Once again, these are just some general guidelines; they are far from ironclad, and they are certainly no substitute for logical and critical thinking. In fact, you really can't use one without the other. Have fun!
www.tektonics.org
500
Notice - The featuring of a particular article does not constitute endorsement of every single item or point of view contained therein by each and every member of TheologyWeb leadership. We strive to have a varied cross-section of representations of differing opinions on secondary Christian issues. The only requirement for the featuring of a particular article is that said article must not contradict the essentials articulated in the TheologyWeb statement of faith found here in our Mission Statement (http://www.theologyweb.com/forum/mission/)or be blatantly offensive to the Christian worldview of the site Owners.
Notice – The ministries featured in this section are guests of this site and very often not active members of debate forums. Additionally, this area is frequented and highlighted for guests who also very often are not acclimated to debate. As such, the rules of conduct here will be more strict than in the general forum. This will be something within the discretion of the Moderators, but we simply ask that you conduct yourselves in a manner considerate of the fact that these ministries are our invited guests. You can always feel free to start a related thread in general forum without such extra restrictions. Thank you.