Open Source Software Implementation Case Study – What is "Discovery"? – Vendor Ecosystem

What is "Discovery"?

• Catalogues...
• Next-generation catalogues...
• Discovery systems...

Vendor Ecosystem

Vendors

• Innovative Interfaces (III)
• SirsiDynix
• Ex Libris
• Proquest, Ebsco, etc.

Vendor relationships

• Software
• Metadata
• Publishing and Open Access

University of Alberta Context

• SirsiDynix Symphony
• Ex Libris SFX
• Database of Databases
• LibGuides
• Relais
• Various homegrown or purchased/licensed systems

Open Source Ecosystem

Contributing factors

• Rise of open source in general (Linux, 1991)
• Increase in computing power, decrease in size (Moore's law)
• Skill development
• Knowledge development
• Budgets and administration

What's out there?

• Operating systems
• Web browsers
• Programming languages and libraries
• Web application frameworks
• Library software (Koha, Evergreen, DSpace, etc)

Discovery Systems, IRs, DAMs

Discovery Systems

IRS (institutional repositories)

Digital Asset Management System (DAMS)

Project Management

• How much is too little?
• How much is too much?
• When do you do it?
• Artefacts
• Tools? Workflows?
• Monitoring
• Knowing when you're done

Requirement Gathering and Design

Team formation

• Scrum recommends a team of between 3 and 9 people (ideally 7)

• Technical skills (programming, modeling, system administration
• Library skills (metadata, description, search/retrieval
• Soft skills (project management, documentation, relationships)

Workflow

• Project management
• Development
• Testing/Staging/QA
• Production

Tools (my favourite part)

• Editor (Sublime, Vim, Emacs)
• Plugins, libraries
• Git/GitHub
• Tmux
• Unix/linux/mac command line

Infrastructure and Hardware

• Again, requirement gathering and design
• Development infrastructure vs. test and production
• Server clusters
• Virtualization
• Local or hosted?

Our discovery application

• Application testing/staging: 2 servers
• Index testing/staging: 1 server, 3 shards
• High-availability proxy (HAProxy): 2 servers
• Front-end application (Ruby on Rails): 3 servers
• Index proxy (solrcloud): 2 servers
• Index manager (zookeeper): 3 servers
• Index shards (solr): 3 servers
• Index shared among all Solr applications (e.g. Hydra)

The Application

• "Blacklight"
• Ruby on Rails
• Solr index
• ruby-marc
• Custom ruby programming
• API calls
• Metadata work

What is "Stack"?

• Data structure
• Software stack
• Hardware stack

Deployment processes (Application)

• Capybara

• Tag release in GitHub
• Jenkins runs tests
• Code packaged (RedHat RPM)
• Deploy to test
• Manual QA
• Deploy to production

Deployment processes (Index)

• Scheduled (cron)
• Full Symphony data (monthly)
• Incremental Symphony data (nightly)
• Ejournals from SFX (nightly)
• Databases from LibGuides (nightly)
• Index data on test
• Manual QA
• Push new indexes to production