Moving Research Online header image
 

About

In response to the COVID-19 crisis, laboratories around the world have ceased in-person human data collection. In principle, many of these laboratories could run their studies online. With over 4,000,000,000 people online, most populations of interest are reachable.

This workshop serves to help laboratories get their research online, covering the key technologies necessary as well as best practices when designing for research on the Web. We will work with the jsPsych and Pushkin free and open source framework, optimized specifically for building an online laboratory for psychological studies.

Workshop details:
June 30th-August 13th, 2020
Where: Online (link TBA)
Cost: Free
Register

Schedule

Session 1: Intro to jsPsych
Presentation: 1-3:30pm EST, Tuesday, June 30th, 2020 [Click for Session 1 video]
Q & A: 1-5pm EST, Thursday, July 2nd, 2020, hosted at: https://github.com/jspsych/jsPsych/discussions

In this session, we will cover the basic technologies necessary to build an online experiment, including:

  1. An introduction to HTML, CSS, and JavaScript (the most common basic building blocks of a web page)
  2. JavaScript's data types and syntax
  3. jsPsych and how to build a basic "Hello World" page
  4. How to use your browser's developer tools to debug

We will cover how to build a lexical decision experiment, containing instructions, lexical decision trials, and a concluding debriefing page. This includes understanding the nature of jsPsych's plugin-based framework, options for creating and adding trials, and where to go for more information.

Relevant resources:
Suggested code editors:

Session 2: Design, Best Practices, & IRB
Presentation: 1-3:30pm EST, Tuesday, July 7th, 2020 [Click for Session 2 video]
Q & A: 1-5pm EST, Thursday, July 9th, 2020, hosted at: https://groups.google.com/g/online-experiments

In this session we will cover key best practices for designing an online experiment. Topics we will cover include:

  • How do I most effectively incentivize subjects to provide high-quality data?
  • How do I best ensure subject compliance & attention?
  • How good is timing accuracy, and how does this constrain experiment design?
  • How long should subject participation last?
  • If I’m used to observing subject behavior with my own eyes, how can I run online studies?
  • How do I restrict participation?
  • Should I restrict participation?
  • How do I address obtaining consent, protecting privacy, etc?
  • How do I work with an IRB that is not familiar with online studies?


Session 3: Advanced jsPsych
Presentation: 1-3:30pm EST, Tuesday, July 14th, 2020 [Click for Session 3 video]
Q & A: 1-5pm EST, Thursday, July 16th, 2020, hosted at: https://github.com/jspsych/jsPsych/discussions

Following up on Session 1, we will dive deeper into harnessing the features of jsPsych to be able to create more complicated study design features necessary for most research studies; this includes dynamically-generated stimuli, performance-based branching, and complex condition randomization and sampling. Core topics will include:

  • Functions as parameters (e.g. practice trials show correct/incorrect stimulus based on previous response)
  • Timeline variables: randomizing and sampling (e.g. repeating the same type of trial with differentsets of stimuli)
  • Conditional and looping behavior (e.g. practice trials repeat if accuracy is below threshold)
  • Working with jsPsych data (e.g. showing a performance summary at the end of the task)
  • A tour of plugin code and how to create plugins

Relevant resources:

Session 4: Massive Online Experiments with Pushkin
Presentation (Session 4, Part 1): 1-3:30pm EST, Tuesday, August 4th, 2020 [Click for Session 4, Part 1 video]
Presentation on Designing Experiments and Q & A (Session 4, Part 2): 1-4pm, Thursday, August 13th, 2020 [Click for Session 4, Part 2 video]
AWS Deployment Presentation and Q & A (Session 4, Part 3): 1-4pm, Thursday, August 27th, 2020 [Click for Session 4, Part 3video]

 * Note: Updated dates! 

Pushkin provides a customizable, scalable ecosystem for massive online psychological experiments. In this session, we will cover how to set up your online Pushkin-based laboratory and the included features, including:

  • Providing participation incentives, such as results shareable on social media or game-like features such as earning badges for participation (for users who create an account on the website)
  • Scaling to balance spikes in website traffic
  • Storing results in a database
  • Robust real-time backups
  • Supporting mobile, tablet, and desktop devices
  • Enabling user accounts for longitudinal studies and boosting participant return rates (e.g. for collecting data for a single participant on multiple studies, with the potential to cross-analyze results within subjects)
  • Supporting a forum, where participants may discuss questions and provide feedback on various aspects of the experiments
The general structure of this session will cover:
  1. Introduction: What is Pushkin?
  2. How to initialize a Pushkin-enabled project
  3. Learning how to launch Pushkin locally
  4. Exploring the anatomy of Pushkin and how to perform some basic customization
  5. Integrating an existing jsPsych experiment
  6. Version control and using basic Git commands to save work to the online repository
  7. Discussion of how to host a Pushkin website on an external server through Amazon Web Services (AWS)

Relevant resources:

Session 5: Concluding Q & A
3:30-5pm EST, Thursday October 1st, 2020

This concluding Q & A will provide an extra opportunity to ask questions about any of the materials covered in the previous sessions, as well as anything else that might arise.

Relevant resources:

Team

Joshua Hartshorne photo
Joshua Hartshorne
Assistant Professor, Boston College
[Website]

Joshua Hartshorne is an assistant professor of psychology at Boston College, and the founder of GamesWithWords.org. He is broadly interested in human behavior, with a particular interest in language and inference. By "inference," he means the application of learned information to make predictions about a novel situation.


Joshua R. de Leeuw photo
Joshua R. de Leeuw
Assistant Professor, Vassar College
[Website]

Josh de Leeuw is focused on the development and application of Internet-based tools for the study of human cognition. He created the widely-used jsPsych experiment software for conducting behavioral experiments online. He uses these tools to study how people learn and represent new information.


Becky Gilbert photo
Becky Gilbert
Investigator Scientist, MRC CBU, University of Cambridge
[Website]

Becky Gilbert is a post-doctoral scientist and web developer at the MRC Cognition and Brain Sciences Unit in Cambridge, UK. In addition to web-based experiment methods, she's interested in speech perception, comprehension, and short-term memory for speech.


Jennifer Rodd photo
Jennifer Rodd
Reader in Cognitive Science, University College London
[Website]

Dr. Jennifer Rodd’s research focuses on how people are able to understand language. What are the cognitive and brain processes that are necessary to understand another person’s speech? For instance, understanding a sentence is more than simply understanding the constituent words — these need to be integrated into a larger conceptual framework and this very often involves resolving ambiguity in the meaning of words. People use a number of different strategies to do this and Jenni’s work aims to identify these, to understand how they work mechanistically, and to explore their underpinnings in the brain.


Constance Bainbridge photo
Constance Bainbridge
Web Developer, Boston College
[Website]

Constance is a web developer, psychology researcher, and musician with a background working with jsPsych and Pushkin through the Music Lab at Harvard University. Constance is beginning a PhD at UCLA in Communication in Fall 2020, and is passionate about studying how we can harness creative and digital tools to communicate our moods and identities to connect as social beings.


Parker Robbins photo
Parker Robbins
Research Assistant, Boston College

Parker is a postbaccalaureate research assistant in the Language Learning Lab at Boston College. He is interested in using web-based experimental methods to bring a larger and more diverse pool of research participants to language science. In particular, he would like to use these methods to investigate the morphology and syntax of understudied languages and to contribute to the language documentation efforts of endangered language communities.

NSF logo
This workshop is made possible by support for Linguistics and Perception, Action, & Cognition at NSF (#2029637 & #2029640).