Hello, and welcome to our in-progress document about Retrospective Methods put in service of learning.
Generally, a retrospective, affectionately called a "retro", is a method of soliciting feedback from participants of a system in order to improve the system for next time. In the context of Hyperlink, retros are dedicated conversations between learners and facilitators about the course they're going through together, how it has worked, and how it can be improved.
We encourage course creators to dedicate time in their courses for a retro. Every cohort of a course is an experiment shaped by all participants, and what you learn can improve the course in important ways. Getting good feedback from learners is a key part of making sure that the course is always evolving in the right direction.
Qualities of a good retro
Goals
Set up explicit, but open-minded goals for your course at the beginning of the learning journey.
I don't believe that you need to define hard quantifiable goals, though that is one way of doing it. Success in learning can have many definitions, and you can set broad, qualitative goals too.
Some examples:
- This will be a success if the course rekindles my love of drawing
- This will be a success if learners develop a new practice to take with them going forward
- This will be a success if learners broaden the ways they think about a particular subject
Investigations into Why
A good learning retro will allow you to explore your outcomes more deeply than a binary "pass-fail" evaluation.
That your goals were fulfilled or not is less important than why. I find that where goals can be vague and broad, the why is best when as specific as possible.
What particular explanations, activities, or methods made the most sense? Which particular ones didn't fit quite right? What was it about those things and their contexts that were successful or not?
A quick note: Finding out what was successful is just as important as what could be improved! Knowing what works will let you infuse more of that into everything else.
Documentation and Next Steps
Be sure to document each retro somewhere and refer back to them when you're improving the course for next time.
We often encourage creators to make their retros public. This allows future course creators to learn from you, and provides future learners with valuable insight into the course, and how you run it.
Important: be sure to respect learner privacy — don't post learner work, words, or personally identifiable info without explicit consent.
Retro Methods: a VERY non-exhaustive index
Exit Survey
An exit survey is one of the most popular and low effort ways of collecting feedback after a course concludes. Simply create a form (in Google Forms or similar) with questions on how the course went, and send it out in an email.
PROS
- Easy to do, and you can whip one up last minute if you forgot to bake a retro method into your course, or you ran out of time
- Doesn't take up precious in-session time
- Can be anonymous, which may allow people to be more honest
CONS
- Difficult to get all (or even most) of your learners to respond, and making your form quick and easy to answer may sacrifice response quality for quantity
- Doesn't benefit from group discussion, which we've found often leads to richer feedback
- Hard to ask for clarification or expansion on feedback you receive
To make your survey straightforward and easy for participants:
- Limit to only 1-2 open ended questions
- Rely more heavily on multiple choice or rating ("how likely are you…") questions
- Try to ask 7 questions or fewer
- Keep the questions themselves brief
Asynchronous Group Discussion
Make a forum post with some guiding questions, and encourage learners to reply with their feedback.
Here's a boilerplate set of questions you could ask. Be sure to tweak or add questions that are specific to the things you're curious about!
- What did you expect to get out this course? Did you achieve that goal? If not, how do you feel about the progress you did make in this course (if any)?
- What specific things (moments, techniques, conversations) did you feel were most useful to learning in this course? Why?
- What is one thing you would change or remove?
- What would you have liked to spend more time on? Anything you think we should add?
- Please share other general thoughts, suggestions, anything else you'd like us to know!
PROS
- Group discussions, where people can expand on and reply to others' comments, often leads to better feedback
- Though not anonymous, this method doesn't require learners to confront you directly, which may make some people more comfortable
- Doesn't bias toward extroverted / outspoken people as much as a synchronous method can
CONS
- It's hard to write good, unbiased, non-leading questions (entire professions exist to write and ask good questions for a living!)
- As with exit surveys, getting people to a respond once the course is over can be difficult; one solution is to give learning time to write during class, though this can eat into session time
- Unlike with exit surveys, you'll typically need to ask more open ended questions, and some learners may find written responses fatiguing
Synchronous Group Discussion
Carve out 15-30 minutes in your last session to have a loosely structured group discussion about how the course went.
You can start by asking the same questions in the async retro section above, to give learners some structure and to fall back on if the discussion lulls. But you don't have to stick to those questions. If someone says something interesting, or the group wants to go in a particular direction, let the conversation take its natural course.
There are lot of interesting ways to structure synchronous discussion, and each come with their own list of pros and cons. This book is a good place to find a variety of options!
PROS
- Most dynamic and least scripted method, which you can direct, and can reach topics that you may not have expected but lead to valuable feedback
- Also the most group oriented, with potential for people to riff off of each other's ideas and talk about the why rather than the what
- Face to face offers a lot more information (like body language, and tone of voice) than what is communicated in text; conducting a sync retro may provide you with more context
CONS
- You're very involved in this conversation, which may discourage learners from being totally open with their comments; be sure to create a space that feels safe, and be vulnerable and respectful of potentially negative comments
- People who prefer to think long and hard before speaking, or who are less likely to insert themselves into conversations, may fall between the cracks; you may want to follow up with an async option or structure the discussion to ensure everyone has a chance to speak
- Can take a lot of in-session time, which you may not have
Critique or Final Review
Have participants keep artifacts of their progress organized somewhere. If these artifacts are collaboratively made, you (the facilitator) may have to organize and/or synthesize everything in a shared location. Artifacts may include:
- Journal entries
- Notes, collaborative or otherwise
- Small projects or assignments
- Incremental snapshots of a longer term project
In the last session, hold a final review of all the artifacts made. This can be a feedback session, discussion, or simply a celebration of the work you've done together!
PROS
- This method is not separate from the course material; it's not only a retro but also a direct vehicle for learning
- Encouraging consistent, continuous work over the course can provide useful structure for learners to keep up momentum and stay accountable
- This also allows you to granularly track learner progress and see what inspired good results
CONS
- Not very explicit about the course structure itself; you'll have to do some extrapolation and experimentation to find what you may need to tweak about the course
- Good feedback takes time to do well — be sure to budget a lot of time to talk about the work!
- You'll need to have a project or assignment based element in your course structure
Incremental Feedback
Identify checkpoints throughout the course (every session, after difficult exercises, in the middle, etc.) to pulse check with your learners.
PROS
- You're more likely to get specific feedback on exercises, prompts, discussions, etc. while the experience is still fresh
- Asking early in the course, and regularly throughout, will give you more space to course correct should you need to
- Consistent reflection on their progress can also be good for learners
CONS
- It can feel like too much time taken in feedback
- You won't get overall feedback on the course as a whole; consider additionally doing a short, more holistic retro at the end of the course
Others?
Have you ever engaged in a retro of your learning? What method did you use? How did it go? Leave a comment — let's retro your retro and add it to this list!