What Can Academia Learn – from Open Source – Kathryn Huff. February 2, 2015
What Can Academia Learn – from Open Source – Kathryn Huff. February 2, 2015
0 0
town-hall
A talk for the UW eScience and GitHub Academia Town Hall panel on "What Can Academia Learn from Open Source"On Github katyhuff / town-hall
What Can Academia Learn
from Open Source
Kathryn Huff. February 2, 2015
Academia Town Hall hosted by UW eScience and GitHub
\[\sigma(E,\vec{r},\hat{\Omega},T,t,x,i)\]
Academia is...
- a community
- engaged in expanding knowledge
- through education
- and research.
Or is it?
Science
- builds and organizes knowledge
- tests explanations about the universe
- systematically,
- objectively,
- transparently,
- and reproducibly.
Otherwise it's not science.
Science relies on
- peer review,
- skepticism,
- transparency,
- attribution,
- accountability,
- collaboration,
- and impact.
Since the 6th century BCE, academic science has been perfecting these tenents.
Open source software is now superior at all of them.
Peer Review, Skepticism, and Transparency
“ Organized Skepticism. Scientists are critical: All ideas must be tested and are subject to rigorous structured community scrutiny.” - R.K. Merton, 1942 “ The scientific method’s central motivation is the ubiquity of error—the awareness that mistakes and self-delusion can creep in absolutely anywhere and that the scientist’s effort is primarily expended in recognizing and rooting out error. ” - Donoho, 2009.Peer Review For Code
- Good: divide and conquer
- Better: shared repository
- Best: peer-reviewed pull requests
Error Detection
- Good: show results to experts
- Better: integration testing, pull-request code review
- Best: unit test suite, continuous integration
Analysis
- Good: pencil and calculator
- Better: spreadsheets, matlab, mathematica
- Best: scripting, open source libraries, modern programming language
API Design
- Good: single block of procedural code
- Better: separate functions
- Best: small, testable functions, grouped into classes, DRY
DRY: Dont Repeat Yourself. Code replication is bug proliferation.
Attribution and Accountability
“ But what if they scoop me? ” - Someone in this room, probably.Publishing First
- Good: share data once all possible papers have been published
- Better: share data as soon as there is a pre-print
- Best: share, with a license, while working, if not sooner
Congratulations: your online repository history is an insurance policy against theivery.
Producing Quality
- Good: being accountable for each paper you publish
- Better: being accountable for the released version of the code
- Best: being accountable for each line of code.
Accountability: git tracks each commit, on each line, for provenance and accountabiliy.
Collaboration and Impact
“ Two of the biggest challenges scientists and other programmers face when working with code and data are keeping track of changes (and being able to revert them if things go wrong), and collaborating on a program or dataset. ” - Wilson, et al. 2014. “ If a piece of scientific software is released in the forest, does it change the field? ”Teamwork
- Good: weekly research meetings, year-long tasks
- Better: daily conversations, month-long goals
- Best: agile development, pair programming, issue tracking
Distribution Control
- Good: "email to request access"
- Better: license file
- Best: license file, citation file, DOI, forkable online repository
Community Adoption
- Good: none, internal use only
- Better: online repository, developer email online
- Best: issue tracker, user/developer listhost(s), online documentation
Extending Software
- Good: hand over a zip file, theory paper
- Better: rely on comments in code, example input file
- Best: version controlled repository, automated documentation, test suite
Unique Issue in Nuclear Engineering
Export control is serious.
Papers!
- Blanton, Brian, and Chris Lenhardt. 2014. “A Scientist’s Perspective on Sustainable Scientific Software.” Journal of Open Research Software, Issues in Research Software, 2 (1): e17.
- Donoho, David L., Arian Maleki, Inam Ur Rahman, Morteza Shahram, and Victoria Stodden. 2009. “Reproducible Research in Computational Harmonic Analysis.” Computing in Science & Engineering 11 (1): 8–18. doi:10.1109/MCSE.2009.15.
- Goble, Carole. 2014. “Better Software, Better Research.” IEEE Internet Computing 18 (5): 4–8. doi:10.1109/MIC.2014.88.
- Hannay, J. E, C. MacLeod, J. Singer, H. P Langtangen, D. Pfahl, and G. Wilson. 2009. “How Do Scientists Develop and Use Scientific Software?” In Proceedings of the 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering, 1–8.
- Joppa, L. N., G. McInerny, R. Harper, L. Salido, K. Takeda, K. O’Hara, D. Gavaghan, and S. Emmott. 2013. “Troubling Trends in Scientific Software Use.” Science 340 (6134): 814–15. doi:10.1126/science.1231535.
- Merali, Zeeya. 2010. “Computational Science: ...Error.” Nature 467 (7317): 775–77. doi:10.1038/467775a.
- Petre, Marian, and Greg Wilson. 2014. “Code Review For and By Scientists.” arXiv:1407.5648 [cs], July.
- Schossau, Jory, and Greg Wilson. 2014. “Which Sustainable Software Practices Do Scientists Find Most Useful?” arXiv:1407.6220 [cs], July.
- Stodden, Victoria. 2010. “The Scientific Method in Practice: Reproducibility in the Computational Sciences.” SSRN Electronic Journal. doi:10.2139/ssrn.1550193.
- Wicherts, Jelte M., Marjan Bakker, and Dylan Molenaar. 2011. “Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results.” PLoS ONE 6 (11): e26828. doi:10.1371/journal.pone.0026828.
- Wilson, Greg, D. A. Aruliah, C. Titus Brown, Neil P. Chue Hong, Matt Davis, Richard T. Guy, Steven H. D. Haddock, Kathryn D. Huff, et al. 2014. “Best Practices for Scientific Computing.” PLoS Biol 12 (1): e1001745. doi:10.1371/journal.pbio.1001745.
Acknowledgements
- GitHub and UW eScience!
- Nuclear Science and Security Constoritum
- Berkeley Institute for Data Science