(Astute readers will note some similarities between this and my upcoming BibleTech talk. But the audiences are quite different, so the content will be too. This talk will provide “a practical case study on semantically organizing reference material to support search and navigation, using a controlled vocabulary.”)

Abstract

Encyclopedias and other subject-oriented reference books frequently present the same content using different names: and users often look for this information using other names altogether.

The Logos Controlled Vocabulary (LCV) organizes parallel but distinct content in the domain of Biblical studies to integrate reference information and support search, discovery, and knowledge management. The LCV captures

• preferred and alternate terminology
• inter-term relationships
• term hierarchy
• linkage to other semantic information

The initial version of the LCV (now shipping in the Logos digital library platform) comprises some 11,000 terms, and continues to grow as more reference works are added. It also provides the backbone of http://topics.logos.com, a website for user contributions to terminology and content.

This talk will describe the building of the LCV, how we’re using it now, and how we plan to use and extend it in the future.

Keywords: controlled vocabulary, publishing, reference, terminology, SKOS

Some time back, Tim O’Reilly (The Architecture of Participation) echoed and applied some observations from Dan Bricklin (the Cornucopia of the Commons) about the architecture of Napster and  other significant web-based systems. The individual details are well worth reading, but here’s the summary form. There are several common models for how to build large datasets that are valuable to people:

1. Pay people to build it (Bricklin calls this “Organized Manual”). Examples include the original Yahoo! directory of the web, and the Encyclopedia Britannica. There’s an variant that represents smart algorithms rather than just human effort (Bricklin: “Organized Mechanical”): this is how Google has built its indexes. But it still represents a significant monetary investment by somebody who probably expects something in return.
2. Get volunteers (Bricklin’s “Volunteer Manual”): Wikipedia is the preeminent example here, along with Linux, the Open Directory Project, and a great many open source projects. People do this work because they value the end result, and the project coordinates and magnifies those efforts.
3. Architect in such a way that individual self-interest creates collective value.

Napster (the original peer-to-peer version) was proposed by Bricklin as a prime example of the third model: simply by listening to your music (within the Napster ecosystem), the default settings meant you were also sharing that music with everybody else. Quoting Bricklin:

What we see here is that increasing the value of the database by adding more information is a natural by-product of using the tool for your own benefit. No altruistic sharing motives need be present, especially since sharing is the default.

This is Bricklin’s Cornucopia of the Commons (an allusion to Garrett Hardin’s Tragedy of the Commons): a system designed in such a way that use brings overflowing abundance.

(You might think blogging and twittering are like this, but they’re not. Nobody tweets because it has direct, inherent value to them: instead, it’s an outgrowth of a narcissistic, self-centered open, generous belief that what i say might have value to others. Few of us would do it if nobody else was listening. )

Models for Data Creation In Biblical Studies

All that (and Napster!) is now history, and i don’t want to get distracted by the peer-to-peer model that made Napster so powerful (Bricklin argues that’s not the reason it succeeded), or the legal issues that led to its demise. Instead, i want to reflect here on how these principles apply to Biblical studies and software.

With Logos 4, we’ve launched a major expansion of our Biblical Knowledge, by expanding Biblical People, adding Places and Things, and building around the large set of concepts we call the Logos Controlled Vocabulary. This was accomplished through the Organized Manual method: we paid a bunch of people (me included) to architect and populate this data, in a major development effort that stretched over several years. You could view the vast network of links that make Logos more than just a collection of texts as an extension of the same principle (through the resulting software program doesn’t look so much like a database). It represents literally hundreds of thousands of hours of effort in book markup and design, along with lots of “Organized Mechanical” algorithmic work.

There are also lots of examples of Volunteer Manual projects related to the Bible. The Sword Project is like Linux for Bible software. e-Sword has a smaller group of developers, but the same framework of a volunteer effort which is given away. Open Scriptures is building a platform and API for others to use in building Bible-based applications. Web 2.0 efforts like YouVersion let people tie their reflections directly to the Biblical text, and numerous projects have sprung from the Wikipedia mold like Theopedia. My own SemanticBible projects are much more limited, but in a similar spirit.

Logos has been active with the Volunteer Manual approach as well. The Logos Topics website combines our Organized Manual data and architecture of topics with user-contributed extensions of additional terminology, links within Logos, and even links to other websites. This lets us do some neat things like extending the desktop application content through user contributions on the web. Like Wikipedia, these are altruistic contributions from people who want to share their knowledge with others.

Sermons.logos.com works in a similar fashion: if you’re a pastor who writes down your sermon, and you’re willing to upload and share it, lots of others (both on the web and in Logos software) can benefit from what you’ve created. This is closer to the Cornucopia of the Commons model, but it’s still a voluntary and indirect process: my sermon doesn’t get shared as a natural by-product of my preparation activity.

The Cornucopia and Bible Study

The interesting question to me is how to achieve the third model, where my own use of a tool provides a direct benefit to others through a network, not because i’m behaving altruistically but simply because the system is architected to work that way. This is closely related to the whole Web2.0 meme (can it really have been five years already?!?) of “software that gets better the more it gets used.”

One thought: lots of web sites use RefTagger to provide a nice pop-up of Bible text for their readers, a benefit that enriches the experience of visitors to their site. Twitter users can similarly use ref.ly to shorten Bible references, which, like RefTagger links,  in turn resolve to references on Bible.Logos.com.   Could those links be converted into data indicating, for example, the relative popularity of different verses, and then displayed back to users?

Aggregating users’ operation of Logos software (in a suitably anonymized fashion, of course) could also provide data on the most popular resources, searches, and topics, which could then be turned around into recommendations (”Looking for a Bible dictionary article on ‘marriage’? Here are the ones our users have found most useful ….”).

But none of these seem to me to accomplish the full promise of the Cornucopia of the Commons. There has to be more here than simply harnessing popularity (though sites like Digg and del.icio.us have shown how useful that can be). I’m still trying to imagine what data sets could be created by people who are already committed to Bible study, as a normal outgrowth of what they do anyway. Any thoughts? Please share a comment.

I’ll be giving two talks this year: here’s my abstract for the first one, on the Libronix Logos Controlled Vocabulary.

The Libronix Logos Controlled Vocabulary (LCV) organizes content from multiple Bible dictionaries to integrate information across the Logos library. As a controlled vocabulary, the LCV identifies, organizes, and systematizes a specific set of terms for indexing content, capturing inter-term relationships, and expressing term hierarchies. Like other kinds of metadata, this infrastructure then supports applications in search, discovery, and general knowledge management. The initial version of the LCV (shipping now with Logos 4) comprises some 11,100 terms, and continues to grow as more reference works are added. It also provides the backbone of http://topics.logos.com, a website for user contributions.

This talk will describe the building of the LCV, how we’re using it now, and how we plan to use and extend it in the future. This includes some interesting new capabilities for machine learning from existing prose content. For example:

• what are the prototypical Bible references, names, or phrases used to discuss a topic?
• can we learn anything about the importance of topics by looking at how much is written about them, how many dictionaries cover them, and other kinds of automated analysis?
• what knowledge can be gleaned from the topology of terminology linkage (what links to what)?

… develop a website, “The World of the Bible: exploring people, places, and passages.” The site is intended for general audiences and will share scholarly views and encourage critical engagement with the Bible, including its ancient contexts and interpretive legacy.

We encourage you to share this survey with people who are not bible scholars—your students, perhaps, or friends and family. The goal is to gain a diverse representation of our intended audience and to assess their current level of familiarity with and interest in the Bible.

Please feel free to post this link in your blog or webpage.

Here’s the link to the survey: if you’re in their target group, i’d encourage you to give them some feedback. I’ve had some discussion with the principals, who know about Logos’ work on the Bible Knowledgebase (but we don’t have any official role in the project). This could become a useful resource for translating some of the scholarly work on Biblical studies to a wider audience.

(Hat tip: Mike Heiser’s Naked Bible blog)

• Technology in the Bible provides an overview, including some metaphors and domain-level groupings (medicine, agriculture, military, etc.). I found his collection on God’s use of technology particularly interesting.
• Biblical References to Technology catalogs many verses with a brief comment on the technology involved.

If you have Logos 4, you can easily play along using the Biblical Things feature (brief tutorial video), which provides a comprehensive list of references for all the physical, depictable artifacts of technology (though not more abstract things like metal refining techniques).

This kind of broad study across the whole of Scripture can provide new perspectives on things that, in their immediate context, often go right by us.

Logos 4 sports an entirely new interface to make it easier than ever to find what you’re looking for and keep your study space organized and effective. There’s a wealth of new, visually oriented resources, and better controls for working through the enormous space of resources Logos makes available. There’s even an iPhone app for no extra charge!

That’s the marketing view (and i stand behind it). But this means much more to me on a very personal level. It’s been almost 3 years since i came at Logos, and this will be the first time most of my work has seen the light of day. Specifically, Logos 4 contains the work of my colleagues and me in several new areas:

• Biblical People, which organizes information about the 3300 individuals, groups of people, and deities named in the Biblical text. It includes a comprehensive list of references, their family relationships, links to dictionary articles, and links to related items. It also includes family tree and story-based diagrams. And everything is hyperlinked.
• Biblical Places includes all the same kinds of information for 1200 named places from the Bible: cities, regions, even geographic features like rivers and mountains. Along with the data, there are 60 new high-resolution maps commissioned by Logos and covering the major Biblical events, as well as a mega-map that shows all the places together.
• Biblical Things describes the physical objects of the Bible: animals, plants, body parts, clothing, food and drink, and much more, as well as specific items like Noah’s ark and Goliath’s sword and weights and measures. There are more than 1000 objects here, which also bring together thousands of images from across the library.
• There’s also a new collection of high-resolution infographics illustrating different aspects of the Biblical world (and i’m extra proud that the bulk of this work was managed by my wife Donna)
• In additional to regular word search (which is much faster than ever), under the hood is the Libronix Controlled Vocabulary (LCV), working to organize 11,000 different subjects in the Biblical studies literature and coordinating information across the library.

So if you’ve been following my posts on the Bible Knowledgebase … well, now it’s here. I can’t overstate how important i think this is: this is quite literally the first time in the centuries-old history of Biblical studies that this information has been made available in this way. The LCV isn’t quite as visible (yet), but it’s also an important organizing feature that will continue to grow in power going forward.

I hope you’re catching my sense of excitement about these new resources (and this says nothing about all the hard work of my dozens of colleagues in other areas). I hoped i’ve piqued your interest to learn more about Logos 4. It really is a watershed event in Bible software.

Obligatory disclaimer: i work for Logos and highly value what i do there. So i’m not the least bit objective about this. (more detailed disclosures)

I’ve got slides from my talk on the Bible Knowledgebase posted now on SemanticBible: the navigational structure above them isn’t in place yet, but you should be able to follow the link directly to get there. Once again, i’ve used Slidy for the presentation, and that process went a little more smoothly this time (which probably just means i’ve gotten better at it). View the source if you want to see how it works.

[Important note: if you were at my talk and wrote down the URL for the slides, i had it wrong. The correct URL is:

http://semanticbible.com/other/talks/2009/bibletech/BK.html

Yes, i know that Cool URIs don't change, which is why i wanted to make this one adjustment before publishing them, so i won't have to change it in the future.]

At some point there should be audio from the talk posted on the BibleTech site (probably on the BibleTech speakers page, which has links to talks from last year and audio where available). Future Blogos posts on the Bible Knowledgebase will go in my WordPress category of that name (RSS feed here), and will also be tagged with bk if you want to follow along.

So one strong candidate for a BibleTech talk is a review of the BK, a machine-readable knowledge base of semantically-organized Bible data that is linked to Biblical texts to support search, navigation, visualization. The thousands of entities in the BK (people, places, and things, along with their names) have a variety of attributes that are appropriate to their type: people have family relationships, places have geo-coordinates, etc. Relationships between entities support discovery and exploration.
Unlike knowledge expressed in prose (like Bible dictionaries), BK data provides reusable content that can serve a variety of purposes. It also provides an important integration framework for Libronix resources, in the general spirit of Tim Berners-Lee’s Linked Data idea.

Some other topics the talk might address:

• visualizing and learning from the graph of relationships
• BK as an information architecture for other Libronix resources
• challenges in building and using BK
• some specific tools that have proved useful in managing BK development
• a possible future for community participation in BK extension

So now, the audience participation portion of our program:

• would you be interested in hearing a talk like this at BibleTech 2009?
• what aspects are most/least interesting to you?

I’d encourage you to post a comment with your responses.

Today i listened to an interview with Dr. Jim Hendler, now at Rensselaer Polytechnic Institute, but previously at University of Maryland and a key figure in the establishment of OWL during his tenure at DARPA. My comments here are really just a rehash of some things he said much better, and with much more authority (given his history in the field) — but blame me, not him, for what i say below.

The concept of the Semantic Web brings together two different communities , along with their respective priorities and technologies. Many of the disagreements within what looks like a single community are just two sets of people talking about different things (but using similar terminology). The “semantic” part is mostly represented by the Artificial Intelligence community, with interests in careful ontology development, deep reasoning, theoretical correctness, and academic activities. The “web” community has been out there for more than a decade, building the World Wide Web with HTML and lots and lots of data, and is now looking for ways to make it more useful, connected, and extensible.

You can represent these two concerns as two axes on a graph, and many different endeavors tend strongly toward one side or the other, depending on whether they emphasize the “intelligence” dimension, or the “data” dimension. Just a few examples on the data side (that could be multiplied many times over):

• Yahoo plans to start indexing RDFa content (i discussed this a bit in my post about Bibleref and RDFa). As one of the major web players, this adds just a little more intelligence to a lot of data (potentially: users still have to create RDFa markup)
• Freebase is harvesting data from Wikipedia and other sources, and then adding a modest amount of structured relations.
• Talis has their own set of data from a long history of library applications.

On the “intelligence” side would be big ontology development efforts, and academics working on reasoning: Hendler also called out pharmaceutical companies as tending toward this dimension. Hendler’s own bet is that progress is more likely to come from data-side approaches than the hard-core intelligence side (and i think he’s right). He sees the combination of SPARQL and persistent identifiers as two recent developments that are likely to move the field ahead: these are things i’m looking at closely as well in Bible Knowledgebase development (more on the second one to come soon).

• programming in C# (i’ve been spoiled by Python)
• Using Visual Studio as an IDE, including integration with MySQL databases
• Basics of 2D graphics
• Layout algorithms for treemaps (major kudos to the University of Maryland’s Human-Computer Interaction Lab for not only pioneering this area, but even providing open source implementations for people like me to learn from)
• Using the excellent (but rich and hence challenging) Piccolo 2D toolkit for building zoomable user interfaces (also from the UMd HCIL group)
• loading up a variety of Bible data (since visualization requires something to visualize!)

I also have a separate presentation about Bibleref: a Microformat for Bible References, and some related recent developments at Logos that will help make the world of on-line information about the Bible more searchable and usable than ever before. You’ll learn more at the conference about some of our plans in this area.

There will also be time Friday night for “birds of a feather” sessions to informally gather people around topics of common interest. I’m hoping to bring together people to talk about developing common naming conventions for people and places in the Bible. If you’ve been following my posts on the Bible Knowledgebase, you know an essential part of this work is simply identifying and disambiguating named people and places: which Judah, or Zechariah, or Gaius, or Jabneel, is which? I think some simple agreement on identifiers, and principles for constructing them, would make sharing such data much easier, and Logos is prepared to start by sharing our own sets of identifiers. So be sure to find me there if you’d like to talk more about how to make this happen. (By the way, i was tickled to see that my post on the most important person in the Bible was #7 in Logos’s list of the Top Ten Blog Posts for 2007 (most viewed)).

As i told one of the speakers in an email earlier today, i’m feeling a little giddy about what a great conference this promises to be. BibleTech, and the interesting and diverse group of people who are coming, really encompasses all the things that brought me to Logos in the first place, and that define my current professional endeavors as well as my personal interests.

It’s not too late! Come join us this Friday and Saturday at the SeaTac Hilton in Seattle (registration details).