Category: Bible Knowledgebase

Wikipedia has proven to be a revolutionary development in online information systems, through features like user-produced content, the breadth of the subjects it addresses, the ability to rapidly update articles, and too many others to list. But one benefit that’s perhaps more subtle is the way that Wikipedia provides a standard set of targets for hyperlinked text.

I use this all the time for my blog posts: as in this recent example, rather than digressing to explain terms like Python and XPath, i just link these terms to their associated Wikipedia articles. Those who know what those terms mean don’t need to follow the links: those who don’t can go find out, if they choose to, or just plow ahead if they don’t want to bother. This hypertext writing style has become much more common in the last decade (thanks in part to the popularity of blogs), and has even spawned new approaches to written communication, like wikis and hypertext fiction.

Once you’ve begun writing (and reading!) hypertext like this, you don’t want to go back: it’s so much more useful to readers to have the additional resources integrated directly into the text. This leads naturally to hyperlinking other kinds of text: for example, i don’t ever write a Bible reference like Luke 11:2-4 without a hyperlink to the verse itself, usually in the English Standard Version (and if you write Bible references like this, you should go look at the bibleref page to see the right way to create these links!).

That’s all background to a realization i had this morning. I was chatting with Dr. Peter Flint, who directs the Dead Sea Scrolls Institute at Trinity Western University, about my work on the Bible Knowledgebase. Dr. Flint gave an excellent talk in the Logos Lecture series last night about his work on the scrolls, and he visited the office today to talk about other things we’re doing. He was reflecting on how, as a professor, he often provides Wikipedia links for his students as additional on-line resources, and wondered whether the Bible Knowledgebase might someday function that way.

In fact, you can use Libronix that way now, with a little knowledge about keylinks and resources. Here are some sample links (which naturally require that you have Libronix installed locally):

• Biblical People data for Simon Peter (as opposed to Simon from Luke 2:25). Since these Biblical People pages are disambiguated, and include links to Bible passages and related family members, they’re useful “hubs” for starting a study on an individual.
• Link targets as text: libronixdls://report|name=ReportBiblicalPeople|page=ID%3APeter-1 and libronixdls://report|name=ReportBiblicalPeople|page=ID%3ASimeon2-1
• Look up the English term “grace” in your default English dictionary (check your settings under Tools > Options > Keylink, for Data Type = English: by default, it’s probably the Merriam-Webster dictionary).
• Link text: libronixdls://keylink|ref=[en]English:Grace
• Look up the English term “grace” in the New Bible Dictionary.
• Link text: libronixdls://keylink|ref=[en]English:Grace|res=LLS:NBD
• and of course you can link to Bible passages, like this Libronix link to Luke 2:25 in the NIV.
• Link text: libronixdls://keylink|ref=[en]bible:Luke.2.25|res=LLS:NIV

If you’re publishing your own content (like a blog, web page, or wiki) and your readers might be Libronix users, this can make it very easy for them to get to the data you have in mind, enriching the value of your content, and making your readers happier at the same time! You can put Libronix links in MS Word documents as well. See this page on the Logos blog for more information about hyperlinks to Libronix, and about using a double-link style to combine web links with Libronix links. (note: I use “libronixdls://” instead of simply “libronixdls:” because otherwise WordPress mangles the reference: your mileage may vary, but in general either should work.)

(Disclaimer: there’s significant controversy in the academic community about the appropriate role of secondary sources like Wikipedia. While i’m not trying to open that worm-filled can, there’s no question that, as background resources, Wikipedia and other on-line content has changed the nature of information and education.)

Update (12/7/2007): Phil Gons of Logos has taken this idea further and spelled out lots of cool things you can do with it on the Logos blog.

(Post 2 in a series on building the Bible Knowledgebase, unfortunately delayed by a plague of web hosting problems)

The What: BK (Bible Knowledgebase) is reference information about the world of the Bible using Semantic Web standards and tools. The Semantic Web refers to moving from a world of networked pages displayed for humans (HTML, the vast majority of the current World Wide Web), to semantically-characterized information that is machine-readable (and therefore supports a variety of uses like search, browsing, visualization, etc.). Tim Berners-Lee likes to describe it as moving from a web of documents (meant to be read by humans) to a web of data (meant to be read by computers).

Initially, the scope is every named thing in the Bible (people and places are the bulk of the cases, but there are also languages, ethic groups, holidays, and numerous others). Eventually i hope to extend this to unnamed but described entities: for example, the Samaritan woman of John 4 is never named, but we know her ethnicity, where she lived, some people she interacted with, and other facts.

The Why: the Bible Knowledgebase will support

• Knowledge exploration and discovery: just as hyperlinked web pages lead you to new information, linked facts about individuals will lead to other individuals or resources about them.
• Smart (semantic) indexing: for a given passage, you’ll know which John/Mary/James is referred to, not just the collection of individuals who share that name. Searching will provide more precisely targeted results, because reference material will be disambiguated.
• Visualization: rich data sets support graphic displays that given an overview of information that would otherwise be scattered across numerous different passages

The How:

• I’ve designed an OWL ontology that captures an initial set of entity types and relationships between them
• Information from Logos’ Biblical People feature and New Testament Names has been merged into the initial dataset
• Both the ontology and the instance data will be extended to incorporate additional information. There’s no principled stopping point, but i expect to grow BK from its current size of ~100k RDF triples to perhaps 100-1000 times that size.

In the (perhaps unlikely) event that you’re in the intersection between

• Blogos readers, and
• attendees at the Semantic Technology Conference in San Jose next week

i’ll be giving a presentation about this work Thursday morning.

(originally posted 2007-04-02, but then a victim of repeat web site hosting problems, i’m trying again …)
Looking further at the numbers i previously discussed for estimating the ranked importance of Biblical characters by how often and where they’re mentioned, there’s a refinement of the dispersion factor that i like better. It came from comparing the rank of Ishmael.1 (Abraham’s son by Hagar) to Ishmael.2 (who assassinated Gedaliah, the Babylonian-appointed ruler of Judah discussed in 2 Kings 25 and Jer 40-41). In my first ranking, Ishmael.2 (who i didn’t even remember) was ranked slightly higher than Ishmael.1, contrary to my intuitions (and those of every Bible dictionary i’ve checked, measured by the number of sentences used to describe each).

Quantifying your ideas gives you a way to measure how they match your intuitions, and, when they don’t, think about why. In this case, it was immediately obvious: though Ishmael.2 is mentioned a few more times, those mentions are highly concentrated, in a total of 3 chapters (across 2 books). Ishmael.1 is also mentioned in two books, but in 6 different chapters. By incorporating the number of distinct chapters a name occurs in (just a more fine-grained measure of dispersion), their rank comes out more like what i’d expect. Specifically, given weights of

• .6 for frequency (as before)
• .2 for chapter dispersion
• .2 for book dispersion (so the total dispersion weight is still .4, just refined a bit)

Ishmael.1 comes out at #257, versus #285 for Ishmael.2. Here’s the top 50 chart using this metric:

Here’s a graphic to show more clearly how the rankings change with this metric. Red markers above the Blue line are names that have moved up in rank with the revised metric: for example, John the Baptist (John.1) moves from #50 to #30, which seems appropriate. Those below the line are ranked less highly under the new metric (e.g. Jesse.1, who moves from #18 down to #24).

Some other factors that might improve the estimate even further (and remember, this is just an estimate):

• As suggested above, external sources (like Bible dictionaries) are a rich and quantifiable source of judgments about importance: the more words or sentences used to describe an individual, the more important they’re likely to be. By using several dictionaries, you’re not held captive to the biases of an individual work or editorial slant. The key feature here is making the connection between the described individual (often in a numbered paragraph) and the Biblical character: but given a map from individuals to passages (which we have), that ought to be possible with a bit of programming at better than 90% accuracy.
• Though the whole of Scripture is authoritative and inspired, there’s a sense in which certain sections are broader in their implications. For example, anyone mentioned in the first chapters of Genesis should probably get an extra measure of importance: these are the foundational stories of Hebrew and Christian history (and this is another way in which Ishmael.1 is surely the most important one).

Postscript: after drafting the above, but before publishing (making and uploading the charts is still a bit painful), i saw this post at OpenBible.info, suggesting some alternative approaches (thanks for playing!). The first (rank some chapters higher than others) is similar in spirit to my second suggestion above: we both agree, as i’m sure do others, that some parts of the Bible ought to add more weight to this metric. The second suggestion there also proposes a valuable refinement, using association with important people (approximated by co-occurrence in a chapter) to lend importance. I’ll look into incorporating that figure as well.

I had a few goofs in the code that computed the name weights in this post, including a parenthesis typo that caused an operator precedence problem and (egad!) having the weight factors for frequency and dispersion reversed.

Here’s a corrected graph, with a larger version linked behind it.

Jeremiah and Joab have each climbed up 5 ranks, their (relative) frequency and dispersion being somewhat different from their neighbors. Otherwise, most of the top 50 are within one or two places of their previous (incorrect) rank.

I’ve left the original there to maintain my humility :-/

[Update (3/29/2007): there’s an important correction here.]

Too many projects chasing too little time means you have to prioritize (in fact, if you don’t have to prioritize, you ought to question whether you’re doing anything worthwhile at all!). I’ve been spending a lot of time lately looking at the Biblical People data in Logos Bible Software, getting ready to incorporate the work i’ve done on New Testament Names, and then to go beyond that to the Bible Knowledgebase.
While we’ve got a lot of interesting data about characters in the Bible, there’s still not as much as i’d like: so how to prioritize additional development? Well, one practical answer is to assign a weight to each one, then start at the top and work your way down, and stop when it’s time to stop (typically because money, enthusiasm, or both are exhausted). While this isn’t a perfect approach to resource development, it’s a pretty good one, and pretty good is often good enough when you’re in new territory anyway.

Since we’ve got the data that maps people to the passages that refer to them, it’s pretty easy to go through and count. Note an important detail: i really do mean references to people, not just strings. It’s not enough to find the string “Judah” in a verse: you want to know when it’s Judah the person, as opposed to a cover term for Israel or the Southern Kingdom. For hard cases like Judah, the only way to know that is go through verse by verse and decide. For many other cases, while the string is only used to refer to people, there are numerous people with the same name. Zechariah is the Big Daddy here: there are 30 distinct ones in our database (this author found 31, but i haven’t gone through them to determine where the discrepancy lies). So just counting occurrences of “Zechariah” doesn’t get it right either: you have to count all 30 cases differently (for those who like skipping to the end of murder mysteries, the prophet Zechariah, whose prophecies are recorded in the book of the same name, gets mentioned the most). You also have to know which names refer to the same person: Simon is Peter is Cephas, and any instance of those names (that refers to Jesus’ disciple, as opposed to one of the other Simon’s) counts. So you need some real data to be able to do a reasonable job with this computation.

There are a lot of different ways you could count: here’s one. Let frequency be a count of the number of verses that mention a given individual (only counting one for verses like Luke 22:31, “Simon, Simon, Satan has desired to sift you like wheat”, which shouldn’t really count as two observations of Simon’s significance as a Biblical character). Let dispersion be the number of books of the Bible that mention the individual. The intuition here is that, for two individuals with the same frequency, the one that’s mentioned in more books is probably more important, broadly speaking. Normalize each of these by their maximum values (max frequency is 1370, max # of books is 31) just to scale things a little more nicely. Then assign a weight to each of these factors (i used 0.6 for frequency and 0.4 for dispersion) and combine them to get a number between 1 and 0.

Here’s what the graph looks like for the top 50 (there are a total of 2987 men and women), ranked by the composite metric (the green line) that combines frequency (the blue line) and dispersion (the red line). The image is linked to a larger version where you can actually read the names.

While the top names (Jesus, David, Moses, Jacob, Abraham) are no surprise, there are some interesting observations to make from this. First, the composite metric really does change the rankings: Levi is #14 by this method, but #53 if you only ranked by frequency. Likewise, Shaul (King Saul) would be #52 if you only ranked by distribution, because he’s mentioned in just a few books: but he’s clearly one of the most important characters in those books, and so it seems fitting that incorporating frequency boosts him up to #15 in the composite metric rank. You see graphically from where the the red and blue lines approach the cases where frequency and distribution are more equal, and places where they’re farthest apart (Judah’s a good example) where they’re most skewed. Back to the previous point about counting genuine person name instances versus strings: only 99 of the approximately 780 occurrences of “Judah” actually refer to Jacob and Leah’s son, so counting strings would be pretty misleading.

Since names, like many linguistic phenomena, typically follow a Zipfian Distribution (sometimes called a “long tail” or power law distribution), it’s no surprise that the majority (1634 of the 2987) of these names occur exactly once in the Bible, and the 59 most frequent names account for about half of all the name mentions in the Bible. So clearly these top names deserve much more attention than the long tail.

Important disclaimer: i’m not making any claims here about theological or historical importance. That’s a subjective matter, and you’d get different answers depending on your perspective. For example, John the Baptist doesn’t even make the top 50, but by Jesus’ own words in Luke 7:28 “among those born of women none is greater than John.” (ESV) So clearly his importance isn’t measured particularly well by this approach. But as a general approximation to how important different names are across the Bible, this isn’t a bad start. To be completely thorough, you’d also want to count pronoun (“she”) and descriptive (“the woman”) references: but we don’t have data for those yet.