If you’re a Twitter person trying to shoehorn your message into 140-character tweets, you’ll like the fact that this gives you a brief and unambiguous way to both specify a Bible reference and link to the content behind it (the references resolve to the actual verse text at bible.logos.com). Since addressability matters, this is a good thing.

But it has precisely the same utility even if you’re not a Twitterhead (i’m not):

• it clearly marks a string of characters as a Bible reference
• it also normalizes the reference into a form that can be automatically processed

While it’s not quite a microformat, it’s really only a small step away from things like bibleref. In particular, if lots of people start using ref.ly references, it will be possible to process that content and understand things like what verses are most popular.

What’s more, editors that recognize and automatically link URLs (like MS Outlook for HTML-based email, and MS Word) will now automatically make Bible links for you (like RefTagger does for blog posts), as long as you’re willing to tack on “http://ref.ly/” and live with the slightly non-traditional format. You don’t need to know anything about how to make a hyperlink in HTML: just a little extra syntax (14 characters, to be precise) moves these references toward much greater usefulness.

I regularly use tab-delimited files for data wrangling: it’s a nice, lightweight format for table-structured data, and Excel makes a good enough editor for non-programmers to change things without messing up the format. Here’s a simple example, with a set of identifiers in the first column: a typical use case would be that somebody is editing the second column so you can map old identifiers to new ones.

Old	New
Aphek1	AphekOfAsher
Aphek2	AphekOfSharon
Aphek3	AphekOfAram

It’s also very easy to read and write this kind of data in Python:

for row in open('somefile.txt', 'rb'):
    old, new = row.split('\t')
    # do something useful here

So i have a little utility reader module doing only a little more than this, stripping out comment lines, returning a list or a dict, etc., and i use this code all over the place. Then i recently needed to read some CSV (comma separated values) files, and stopped to ask The Question, which every programmer should ask before writing new code:

Hasn’t somebody else solved this problem already?

In the case of reading and writing CSV files, the answer was a quick and clear “yes”: there’s a standard Python module called csv that does just that, and nicely. So, reformatting the earlier data example as CSV would look like this:

"Old", "New"
"Aphek1", "AphekOfAsher"
"Aphek2", "AphekOfSharon"
"Aphek3", "AphekOfAram"


and there’s a nice DictReader method that (assuming your columns are unique and your first row identifies them) makes working with this data even easier.

import csv
reader = csv.DictReader(open('somefile.csv', 'rb'))
for row in reader:
    #do something more useful here
    print row.get('new')

If the first row doesn’t contain column headers, you can supply them to DictReader. This looks like overkill for this simple problem, but once you have multiple columns, need to check values or map them onto something else, or add other logic and processing, life is just much easier with a dictionary structure (for one thing, you get rid of meaningless mystery indexes and stop asking “what the heck is in row[1]“?).

Now comes the embarrassing part: i quickly breezed through the documentation, accomplished my immediate task, and moved on, missing one important detail that i just now (a month later!) figured out. Tab-delimited files are just a special case of a CSV file. My original, tab-delimited file works just the same way, once i construct the reader with tabs (rather than the default of commas) as the delimiter.

import csv
reader = csv.DictReader(open('somefile.txt', 'rb'), delimiter='\t')
for row in reader:
    #do something more useful here
    print row.get('new')

There are a few other gotchas, the most important of which for me is that csv doesn’t handle Unicode. So if you have to read Unicode data, you’re back to reading the data directly, splitting lines on tabs, etc.

The best code is usually the code you didn’t write and don’t have to maintain. No matter how many times i stop and ask The Question, i still don’t do it enough.

• Dostoevsky’s 1984 Saved Him from Our Brave New World (from John Dyer at Don’t Eat the Fruit): reflections from education and learning author Neil Postman on Orwell and Huxley’s contrasting views on information overload.

“The amount of information we intake overshadows what little significant information we do intake.”

• Who Cares About Raw Data? (from Jesse at OpenScriptures): even though it may not be immediately apparent to end-users, data is the lifeblood of applications (a frequent Blogos theme), so it matters a lot.
• On the horizon: Google Wave (from Mark Hoffman at Biblical Studies and Technological Tools). I agree with Mark that Google Wave (which isn’t quite released yet) bears watching as a really interesting new development for collaboration and communication.

In some respects, Twitter seems like another replay of several earlier IT pyramid schemes: first bulletin boards, then websites, then blogs. The earlier adopters get the lion’s share of attention, and the people who follow them get the initial benefit of more information and (maybe) more access. But as the pyramid grows over time, the volume of communication becomes unmanageable. Then we evolve new schemes for managing the flood: bookmarks, feeds and feed readers, tags spaces (hash tags for Twitter), and other meta-schemes. At some point we ought to stop and ask whether the new communications services and the additional complexity and overhead they impose provide enough benefit to justify their cost.

I’m humble enough to acknowledge that many smart and productive people say they get a lot of benefit from Twitter, so maybe i really am missing something by remaining doggedly tweet-free. But the latest issue of eWeek (you may need to download their reader to access it) has a nice review of several enterprise-oriented social communication services (SocialCast, Socialtext, and Huddle) that’s making me rethink my position. These have several distinct differences from Twitter and Facebook that make them more interesting to me:

• Narrower scope of communication: While broad services like Facebook and Twitter provide access to a universe of information, that can become overwhelming, particularly when the universe is talking back at you. Limiting the conversation to what’s happening in our company has a lot of appeal.
• Better business information: getting the right mix between quantity and focus about what’s going on in your company has always been a difficult challenge in my business career. The people who have the most to say are also very busy, and of course they already know what they know: so there’s asymmetry, with a cost to them in producing information while most of the near-term benefits accrue to their listeners.At the same time, there’s no easy way to predict what information might be useful to whom, so generating lots of it makes sense provided there are effective ways to filter it.
• Distributed leadership: personnel at all levels have useful things to offer your company, if you can just break them out of the stovepipes of departmental structure and management hierarchy.

Here’s how i might see this playing out at Logos. Our business structure is largely typical for a software company of our size: there’s a sales and marketing division with some people who travel a lot and who (along with inside sales) provide a lot of our revenue, a customer service group that deals with users and their problems, the programmers who make the application work, a smaller group (where i work) that’s developing new features and data sets, a variety of support and infrastructure people who keep IT, finance, etc. running smoothly, and of course a management team that’s steering the corporate ship. All of these groups have front-line access to information that might help the rest of us do a better, given the right kinds of access.

One of the challenges is to keep the information we all broadcast a little more targeted. While some amount of personal interest makes all the hours we spend at work more fun, too much information about somebody’s disappointment that their favorite team lost, or how much they hate paperwork, the lousy sandwich they had for lunch, etc. would start to make this just like Facebook (and i don’t think having employees reading Facebook during work time is a good strategy for productivity). But i’d love to hear:

• from front-line sales people: a conference sales pitch that really hit home with people; the reason somebody just gave for purchasing our software (or not purchasing it); a complaint from a loyal customer
• from customer support: the common problems that get raised over and over, and that would make make for a much happier user experience if we fixed them upstream
• from support staff: the challenges that we can all help with
• from programmers and from R&D: exciting new discoveries, tips and tricks
• from management: new things we’re learning and thinking, big picture business issues, where we see our business heading

Many of these things get communicated now, just more formally, and therefore less frequently, and to a more selective audience within the enterprise. Moving toward Twitter-style microcontent, just a little more focussed, might provide the right mix to get me tweeting.

The larger question this raises for me is how we might come up with a comprehensive, global score for verses to indicate their importance for a variety of purposes. As the OpenBible post suggests, this depends both on what the source of the data is, but also on what your purpose is and what you mean by “important” (which is certainly different from “popular”, though not completely unrelated).

One useful purpose is ranking verses to present them in response to searches: TopVerses.com is explicitly organized this way, as indicated in this news article about the site. They don’t go into much detail about how they gathered their data, though the scope (37M references scoured from the web) is impressive. But there’s a subtle disparity here: their data is based on counting mentions (citations) in published web pages, but their use case is prioritizing search results, and these may be out of sync. The fact that a given verse is frequently published on the web doesn’t necessarily mean it’s the one you want at the top of the list when you’re doing a word-based search, for example. The other three sources seem perhaps better matched to ranking search results, since they’re derived from searches themselves.

Another key hitch is these endeavors is how to handle range references, both in processing source data and (for search purposes) in handling queries. For example, many Bible dictionaries frequently reference ranges of verses, sometimes extensive, multi-chapter ones. If you’re going to count these, you need to think carefully about how you do the counting so you don’t introduce bias (or, better, you select the bias that’s best suited to your purposes).

For example, in the TopVerses.com ranking John 3.1 is #26, despite the rather plain descriptive content with little obvious spiritual impact.

Now there was a Pharisee, a man named Nicodemus who was a member of the Jewish ruling council. (John 3.1, NIV)

While i can’t be sure, i strongly suspect this high rank is an unintended consequence of  dis-aggregating ranges and whole chapter references from John 3. In fact, scanning top verses by chapter from John, the first verse in each chapter is very often the highest or second-highest ranked, and near always among the top ten. This probably says more about the counting methodology than the significance of those verses in particular. The Bible Gateway study focuses on ranges of no more than three verses to explicit mitigate this problem.

Other Measures of Importance

Moving from popularity to importance, i can imagine several different factors that might be combined to produce a more general importance score:

• citation frequency (based on some corpus). In the TopVerses.com approach, these are web pages, which provides a very large set of observations. A number of other digital text collections would also suit this purpose, and even allow segmentation by genre: for example, you get a very different ranking from the Anchor Yale Bible Dictionary compared to Easton’s (and neither have John 3.16 at the top of the list). See below for more about this.
• search frequency, the basis for the other three sources in the OpenBible.info post. This could be refined further given data on follow-up activities. For example, depending on your application, verses searches whose results are then expanded into a chapter view or followed to the next verse might get a boost compared to those with no further action (this seems like a variant of “click through” rates used in search engine advertising)
• content analysis (context-independent): this could have several different flavors.
  • word count: though John 11:35 gets mentioned more than you’d expect precisely because it’s the shortest verse in the (English) Bible, in general longer verses are more likely to be important. This could be refined further given a metric for important words (but now we’ve introduced a new problem: where does that data come from?), which could be used for weighting the counts.
  • We could do even better if, instead of counting words, we count concepts (and weight them). Assuming we think the concept of HUMILITY is important, we’d want verses expressing that concept to rank more highly, regardless of whether they used a more common word like “humilty”, or a less common one like “lowly”. Converting words to concepts is a difficult challenge, however.
  • Connections to other data also affect importance. In some sense, every verse that reports words of Jesus is probably more important to a Christian than one whose importance is otherwise comparable, which is why we have the convention of printing Bibles with the words of Christ in red (a binary system for visualizing importance).
  • We might even consider negative factors: a lower rank for unfamiliar, hard-to-pronounce names, or “taboo” words.

Unlike TopVerses.com, i don’t see a particular need to provide a unique rank for each verse. If each verse has a score (to simplify the math, a decimal between 0 and 1 is a common approach), you can simply pick the top n verses that fit your purpose, and then order any ties canonically.

Comparing Dictionary Reference Citations

I did a small experiment to compare the most frequent reference citations in seven Bible dictionaries that are incorporate in Logos’s software (so this is citation frequency, not search frequency). I extracted and counted all the references, and then aggregated the counts across all seven: the top 20 references are shown below, along with how many “votes” they received in the OpenBible.info list. In the case of whole chapter references (four of the top ten), i’ve indicated with yes/no whether any verse from that chapter occurs in the OpenBible list.

There’s relatively little overlap between the two lists: only seven of these are in the OpenBible list. Many of these make sense given the different purposes of reference works: for example, Is 61.1 is a key messianic text. The high rank for 2 Ki 15.29 is initially puzzling, but probably results from being commonly cited in discussions of the conquests of Tiglath-Pileser and the Babylonian exile. Overall, this is probably much too small a sample to show the correspondences: i presume we’d find much more overlap in the top few hundred.

Details:

• The dictionaries used were the Anchor Yale Bible Dictionary, Baker Encyclopedia of the Bible, Eerdman’s Dictionary of the Bible, Eerdman’s Bible Dictionary, International Standard Bible Encyclopedia (ISBE), New Bible Dictionary, and Tyndale Bible Dictionary.
• Like the OpenBible.info approach, i took range references with 3 or fewer verses and decomposed them into individual verses, splitting the counts (which is why the aggregate counts are floats rather than integers). Larger ranges were left atomic, which confuses the results further: for example, Ge 1:26 probably ought to be even higher, since the high-ranking chapter reference Ge 1 includes it.
• Some references are undercounted because this method distinguishes BHS and LXX references, but i doubt this materially affects the results.

Conclusions

None of this is meant as criticism of the particular sites mentioned above. I strongly believe that any user-oriented, empirically-based data set is better than nothing, and in most endeavors like this, “the best data is more data”. * But with more data comes more complexity, and i’ve only scratched the surface here in considering some of the different factors.

The key point is this: if we want to measure something, we need to be clear up front about exactly what it is, and also what purpose we hope it will serve. I never stop being amazed at how often “obvious” approaches to data problems produce surprising results.

Their site offers an integrated tool for accessing

• Six different Bibles, along with concordances and the Oxford Bible Commentary
• Reference tools that span nine different sources (The Oxford Companion to the Bible, Oxford Bible Atlas, Oxford Encyclopedia of Archaeology in the Near East, etc.)
• A collection of about 1000 images and maps.

Whether this is helpful to you will be strongly determined by whether you value Oxford’s resources, of course: this is a walled (publisher) garden in that respect. But in my opinion (and those of real scholars as well) there’s a lot of very good material here. They provide an integrated search feature so you can search by keyword across all the reference works, Bible texts, and image resources. You can also limit that search by some general domains like Archaeology, Geography, Material Culture, etc.: that’s a nice way to make the search results a little more manageable in size. 

There’s also a timeline feature, with links to the reference works (like Oxford History of the Biblical World and Companion to the Bible). I found that an interesting way to navigate to relevant information: you can find an item in the timeline (”Manasseh is king of Judah”) and then follow a link to an article on Manasseh.This kind of interface, where you can get to new information without first having to know exactly what you’re looking for, is important for those who aren’t already specialists in the domain of Biblical studies. 

One very helpful feature is that dictionary-type articles have some number of terms hyperlinked: Bible references, of course, but also maps and other articles. So the article on Manasseh links to articles on Chronicles, Deuteronomic History, Ephraim (for the “other” Manasseh: the article combines both), and others. These links are also displayed in a sidebar, not just embedded in the text. This feature is limited somewhat, though, by the fact that the links are only within the current reference. What would make this feature really useful would be to link across their collection, so you could also easily get to e.g. the Bible Dictionary article on Ephraim. Of course, you can do that by going back to the search interface, but that’s some added friction. Also, it would appear that at least some of the links were automatically generated: the prose about Manasseh (son of Joseph the patriarch) links to “Joseph (Husband of Mary)”. 

I found a few things i didn’t like as well. While the “unbundled” content of the reference tools is available, in some cases (for example, the Companion to the Bible) i couldn’t figure out a way to actually read the books as books: i could only get to them as disembodied search results. I suppose that’s all right for strict reference purposes, since search is probably the normal way you’d access them. But since some other volumes (like the Illustrated History of the Bible) can be read sequentially, i don’t quite understand why this isn’t possible for all of them. Another mild annoyance: while they helpfully link Bible references to their text, they only put the chapter and verse in the hyperlink, like this:

Amos 9: 9        

In addition to looking a little weird, it also makes for a smaller (and therefore harder to hit) mouse target. I can understand doing this in a list of references where the book name isn’t repeated: but where the book is adjacent in the text, i don’t see the rationale. I also noticed that some features of the site (like Look It Up) didn’t seem to work quite right with Google Chrome.

These are pretty mild complaints, however. Overall, i’d say there’s a lot of very valuable information available here. The annual subscription price of $295 would seem to best fit serious professional users, however. I’d encourage you to take advantage of the free trial period and investigate it for yourself. 

[This seems like as good an opportunity as any to point out that i followed the good example of my colleague Rick Brannan and created a Disclosures page, to avoid any potential confusion about my financial relationship to the things i write about. In this context, full disclosure: i work for a company that is partially in competition with Oxford's site. ]

Susan Boyle

If i’m honest,  i’d have to admit my first reactions to seeing this rather plain, unemployed middle-aged woman who lives alone in a small village in England were no different than many in the audience (of course, the producers of the show played up this angle a bit). I couldn’t help thinking of 1 Sam 16:7 and how many people i walk by each day who look like nothing on the outside but have a tremendous gift inside, if only i could perceive it.

Our brains are wired to work this way: we make thousands of quick judgments throughout the day based on what we see, and we’d have a hard time coping with the complexity of life without this ability. But precisely because of this tendency, we need to constantly guard against persisting in these perceptions, when instead we should take the time to listen.

[hat tip to my sister-in-law Doreen for passing this along to us: thanks a lot for making us cry first thing in the day!]

Step #1: You can download here a search plugin for Firefox that goes to Bible.Logos.com. Once you’ve installed it, you’ll see something like this in the upper-right corner of your Firefox window.

Type in a Bible reference (or a word or phrase to search for) and hit return to do the search.

Step #2: What i like even better (which is how i found this in the first place): once you’ve installed the plugin, it adds an item to Firefox’s context (”right-click”) menu. That means if you’re on some random web page, and the author wasn’t thoughtful enough to install RefTagger, you can simply select the text of a reference, right-click, and so a search like this, with no typing required:

Just another simple way to reduce the friction in your daily information-seeking …

Of course, other plugins can choose to extend the context menu (as LibX helpfully does, shown clearly in the snapshot above). I thought the Logos plugin did too: apparently not, but i’ll remain hopeful for the future.

I’ve posted an experimental visualization of the events of Holy Week, based on the Composite Gospel Index.

I’d be interested in feedback on the layout: i struggled quite a bit with getting the colored bars to show the quantitative information, but also having links. In particular, i would have liked to have pop-up text on each colored block with the Scripture passage, but i couldn’t figure out a way to do that with RefTagger without actually having the reference text, and that has layout issues for small blocks. No doubt Javascript mavens could do something cooler. But this is done only using XHTML and CSS, so perhaps i get some standards points.

It’s difficult, and sometimes impossible, to determine the precise chronology of the Gospel events, so this is a “best attempt”. I’ve followed the article on Chronology (Libronix link) in IVP’s Dictionary of Jesus and the Gospels in placing the Triumphal Entry on Monday (though of course we celebrate it in church on Sunday).

I’ve got slides from my talk on the Bible Knowledgebase posted now on SemanticBible: the navigational structure above them isn’t in place yet, but you should be able to follow the link directly to get there. Once again, i’ve used Slidy for the presentation, and that process went a little more smoothly this time (which probably just means i’ve gotten better at it). View the source if you want to see how it works.

[Important note: if you were at my talk and wrote down the URL for the slides, i had it wrong. The correct URL is:

http://semanticbible.com/other/talks/2009/bibletech/BK.html

Yes, i know that Cool URIs don't change, which is why i wanted to make this one adjustment before publishing them, so i won't have to change it in the future.]

At some point there should be audio from the talk posted on the BibleTech site (probably on the BibleTech speakers page, which has links to talks from last year and audio where available). Future Blogos posts on the Bible Knowledgebase will go in my WordPress category of that name (RSS feed here), and will also be tagged with bk if you want to follow along.