BiSciCol core software architecture

This simplified UML diagram illustrates the current architecture of the core BiSciCol classes. These classes are responsible for most of the lower-level BiSciCol functionality: interacting with data, working with queries, traversing the BiSciCol data structures, and converting BiSciCol data to other representations, such as XML. Note that this diagram does not depict all relationships or dependencies among the classes. Instead, relationships most important for a general understanding of the code are included, such as inheritance, interface implementation, and aggregation. Further, most private class methods and members are excluded for simplicity.

In general, we've been working to develop an architecture characterized by classes with clearly-defined responsibilities and a high degree of flexibility. We've also tried to develop objects that map naturally to the BiSciCol problem domain. For instance, both BiSciCol data objects and models are represented by corresponding abstract data types. Furthermore, the code that implements BiSciCol objects and data operations is completely separated from the code responsible for converting BiSciCol data to other formats, such as XML.

This is simply a snapshot of the state of the code at this time. Some of the classes illustrated above are incomplete and/or will almost certainly be redesigned. Collaborations between classes will likely change as well. Nevertheless, the diagram shows not only progress we've made, but also many of the design ideas we've been working with.

BioSciCol VertNet Integration Meeting

Recent big news is the funding of VertNet! Given this, some of us got together to discuss ways we can work together towards some common goals.

Location:”Jupiter Cafe”
John Deck (Kolsch)
Aaron Steele (Jupiter Red)
John Wieczorek (Hot Chocolate)

VertNet will publish to PubSubHubbub. BiSciCol can subscribe to the hub and receive updates on what is going on.

JohnD:
- VN can encourage, promote, implement use of GUIDs
- Links to GUIDs of Annotation references (of all kinds like data quality stuff e.g. Creation of WGS84/dec.lat/lng instance of DMS data)
- BiSciCol can help VertNet in augmenting search (relationship graph search)

JohnW:
- BiSciCol is base study for annotation use cases (query on annotation relations?)
- VertNet won’t look at relationships between objects, so BiSciCol could be FTW
- Recommends Twitter dialog between BiSciCol and VN

Aaron:
- Annotations
- PubSubHubbub
- Twitter dialog

More to come. If 2 of the attendees can figure out how to use (and if they really want to use) twitter we can stay plugged in that way (per JohnW's suggestion). Otherwise, we'll be forced to meet over beers and use our speech.

Revised Prototype Implementation Diagram

This is a refined diagram describing our preliminary implementation

Development Update

Brian Stucky and I made a first pass at an online prototype back in mid-February. This prototype featured a text-file triple-store back-end, a bit of javascript, a cool modeling widget, and some JSP glue.

Since then, I've been working on a few things to make this prototype more solid:

1) Implementing a database back-end for the triple-store. I've been working with Virtuoso under an evaluation license and have found it fairly easy to work with. I would prefer a free or more open-source solution but it seems Virtuoso is the best product out there for our needs.

2) Building queries on REST services. Not wanting to make the REST services an after-though to our architecture, I am building the REST services into the our web prototype.

3) Re-building SPARQL queries to be more portable, scalable, and faster. Specifically, the transitive closure functions were previously built using specialized syntax not portable to larger systems. The new syntax is more portable and will be alot faster. E.g:

SELECT ?s ?dist FROM
WHERE
{
{
SELECT ?s ?o
WHERE
{
?s bsc:relatedTo ?o .
}
} OPTION (TRANSITIVE, t_distinct, t_in(?s), t_out(?o), t_min (1), t_max (20), t_step ('step_no') as ?dist) .
FILTER (?o= dwc:spec12345)
}

Hoping to get this new prototype online in the next week or two.

Preliminary Implementation Diagram

An indication of where we are headed with our architecture/implementation. This is a draft form and comments are welcome!

BiSciCol Technical Meeting: University of Florida, 23-24 February 2011

The BiSciCol Technical team had a successful brainstorming meeting over two warm Florida days.

Attendees:

Nico Cellinese (UF), Kate Rachwal (UF), Reed Beaman (UF), Gustav Paulay (UF), Russ Watkins (UF), John Deck (Berkeley), Brian Stucky (Colorado University), Rich Pyle (remotely, Bishop Museum) and our very special guest Steve Baskauf (Vanderbilt University).

The main goal of the meeting was to define the BiSciCol scope, techcnical goals and the model. Happy to report we made great progress. We are now ready to move to an implementation phase and are confident will be able to release a prototype in July 2011. See below a snapshot of out design document.

Purpose of the System

1. Notify all objects that are related to Object X that Object X has changed.

2. Manage relationship definitions between objects that exist in distributed databases.

3. Manage annotations of objects where annotation is treated as an Object relationship.

Technical Goals of the System

1. scalable design

2. easy coding but must be robust

3. expandable to different domains

Envisioning this as a distributed network with structured content defined. Goal is to allow for multiple data sources, each with their own thematic content. The thematic content is built on defining relationships between objects.

The Model

Linkages between objects is unstructured which allows any object to be tied to any other object via the predicate “relatedTo”. This is similar to SKOS “related” but we have made relatedTo transitive. relatedTo does not require objects to be organized hierarchically.

Following are the triples recognized in the network:

:objectId :relatedTo :objectId

:objectId rdf:type :objectType

:relatedTo :hasProperty :propertyDefinitionURI

:objectId dwc:dateLastModified “YYYY-MM-DD”

News and updates

Since our last meeting in Washington, DC we worked out some of the object relationships and how we want to formalize concept/objects such as individuals, specimens, vouchers etc. for the purposes of our network. We also agreed to consider implementing RDF in our design. To this end, a number of people have since purchased and read (hopefully past the introduction) the book "Semantic Web for the Working Ontologist: Effective Modeling in RDFS and OWL". Finally, we agreed to deliver a working prototype of the BiSciCol network in July 2011, implementing data from Biocode, CalPhotos, and UF.

During January and the first part of February, John Deck (Berkeley), Rob Guralnick (Colorado U.), and Brian Stuckey (Colorado U.) have been crafting a technical implementation plan for the BiSciCol prototype. CU has offered a server and Tomcat instance for our prototypes and Brian has been implementing that along with getting the web interface going. John has been working on a model that ties together collecting events, specimens, tissues, DNA extractions, and photographs (more object types later), while incorporating location and modification date. Taxonomy is notably absent for now and we will soon be working more closely with Rich Pyle and Rob Whitton (Bishop Museum). Codebase is Java and inferencing/RDF work being implemented in Jena/ARQ (open source Java).

On the 23rd and 24th of February, John and Brian will be at UF for a technical meeting with Nico Cellinese, Kate Rachwal, Russ Watkins, and special guest Steve Bauskauf (Vanderbilt University) to review the UF implementation and proposed model. We will also work remotely with Rich Pyle and Rob Whitton. More details on this meeting will be posted soon. Given our prototype deadline of July 2011 we have a lot of ahead of us, especially in integrating taxonomic names components but we feel we are on target with progress to date.