I'll use this space to outline some of my techniques and research. You can also find my new teaching blog at https://uwaterloo.ca/scholar/dkchisho/blog
In August 2018 I had the immense pleasure of attending my first live action role-playing game at Bothwell School of Witchcraft. Everyone was given a character to play, and the action unfolded over 3 days based on the group’s collective improvisation skills as well as the groundwork laid by the organizers. I thought I was merely vacationing while indulging in my fascination with a certain fictional British magical boarding school, but it turns out even in a real castle in East Sussex there is pedagogical inspiration to be found.
Bothwell participants play either as a student or as a professor. While applying, I indicated I'd be willing to play a professor, since I literally am one. You can bet I would have added "3 days teaching Magical Defense at Bothwell School of Witchcraft" to my CV if I had been accepted! But as a role-playing novice, I was assigned a first year student character.
Those who were designated as professors had to develop their lessons for three years of students and teach each cohort twice. Each professor had their own distinctive style, and had clearly put vast amounts of work into thinking about their pedagogy. Without collaborating beforehand, and mostly with no teaching experience, these nine strangers from four different countries converged on some common ideas about what makes an effective lesson:
These concepts of effective lesson design transcend disciplines and even provide value within a world of make believe. Perhaps we can remember some of these ideas in our own (albeit slightly less magical) classrooms! And if you'd like to give magical instruction a try, registration is open for Bothwell 2019.
* Magical History explored relations between magical and non-magical populations, Magical Defense taught about the power of casting spells as groups, and Daemonology taught how to recognize and exorcise people who are possessed, all of which played major roles in the overall story development.
I attended the 2017 Actuarial Research Conference (ARC) and was fascinated by a talk that argued insurance companies should consider firearm ownership as a rating factor. Firearm presence has a stronger impact on claims than many other factors that are used. The reason for its absence is not efficacy but politics: there is opposition from groups that have a vested interest in more firearms being sold. I was intrigued and wanted to weave this kind of topic into my teaching.
Around the same time, I was finishing up the Fundamentals of Actuarial Practice modules, which examine actuarial work in a more nuanced and sophisticated way. For example, they discuss what external factors actuaries need to consider, how they set assumptions for models, and how they justify their choices. This reinforced the potential value to think beyond the textbook in my course design.
I taught ACTSC 232, the introductory life contingencies course, in Winter 2018, so this was a perfect time to incorporate some of these new ideas. Thus I developed a segment called "Think Like an Actuary" (TLA) to get students to think about the more complex issues that actuaries face, not just the formulas they need to pass exams.
I included TLA activities in many lectures (having students brainstorm ideas and collecting them on the board), in every tutorial activity (work they could do in groups and hand in, with instructors and TAs there to assist them), in every assignment (requiring them to write a one-page report), and on both midterms and the final exam (with limited time, somewhat more closed-ended questions), so it was truly fully integrated into the course.
Some TLA examples include:
Although students were hesitant at first, asking "so you want me to write a sentence?" and questioning why I was asking them things that were not specifically in the textbook, it didn't take long before students saw the value. They began having spirited discussions and debates, looking at underlying assumptions with a critical eye, considering multiple perspectives, and realizing the nuance of actuarial work. I was pleasantly surprised at the depth of thought that these young students (most only in second year!) were able to come up with, once they were encouraged to.
For part of their final assignment, they had to look back on all the TLA topics they had seen and write a reflective paper about what they found most surprising or interesting. I was overwhelmed by the responses. Students said things like:
I was so inspired by the original ARC 2017 talk that not only did I develop TLA and heavily incorporate it into my course, but I ended up speaking about it at ARC 2018. My slides from that talk are here.
I’m so proud of how much my wonderful students learned, and how deep they were able to extend their knowledge in their first life contingencies course. I look forward to seeing them develop into thoughtful, professional actuaries!
I was invited to give a talk on June 14, 2018 for Dan Wolczuk's seminar for UW instructors, and chose to talk about Assessment Design for Learning. You can view the presentation here and access the slides here. In it, I touch on several of the ways I design assessments in my courses to encourage student learning, including:
Enjoy this glimpse into my teaching and assessment design philosophy!
Back in early 2015, the SAS (Stats and Act Sci) department was having a discussion about CS students in the two required STAT courses (230: Probability and 231: Statistics). While CS students are as strong as other Math students in most courses, they were systematically underperforming in STATs.
The idea that got the most traction was to pilot a special section of STAT 230, which would focus on the CS applications of probability. The hope was that by including more relevant examples, and pointing out the many important tie-ins to later CS courses, the students would be more engaged and performance would improve.
But who could design and teach such a section? I immediately and enthusiastically volunteered! I had many years’ experience teaching probability courses. And to help me develop CS-centric examples I could leverage my many close friends (including my husband) who were UW CS grads working in the tech sector. When I reached out to my network via Facebook, I received over 100 ideas.
The SAS department gave me their blessing to start working on it, so in Spring 2015, I taught the first CS section of STAT 230. To be fair to all students, examinations were identical between sections. Although I wasn’t given dedicated time to develop the new section, I was able to introduce minor changes in that first term, and make small tweaks in each subsequent term. It wasn’t necessary to overhaul the entire course in the first pilot, merely to start the process and then build upon it.
I began by incorporating interesting CS examples of STAT 230 concepts into the lectures, often substituting out toy applications (e.g. balls and urns) with more appropriate ones (e.g. binary trees, databases, sorting algorithms). I wrote CS versions of questions for quizzes and tutorials, and many of these were later added to the course notes. This had the beneficial side effect of providing better breadth of practice problems for all students and providing resources for other instructors of the CS section of the course. By ensuring materials would be available in the future I increased the probability that this differentiated section would be sustainable and my investment in the course would live past my involvement.
In the pilot course I polled students for games they liked, and used the results for my "Stats Weekly Application to Games" (SWAG). We discussed how probability applies to Poker, League of Legends, The Resistance, Hanabi, and more. Each example or application was introduced right when we had just covered the applicable theory in the course; for example, we talked about card-counting in Blackjack right after learning the Hypergeometric and Binomial distributions. In later terms I added a "Machine Learning Idea of the Week" (MLIW) segment including Bayesian classifiers, the "bag of words" algorithm, and ethical use of data.
After the first two terms of the pilot, we conducted a survey. An encouraging 59% of the students said they appreciated the tailored material and 62% said we should continue offering the special section.
After several more terms with CS sections, we then had enough data to reliably draw conclusions about the effect on student performance. While it would be easy to evaluate the change in CS students’ marks, this would be statistically invalid, since there could be other factors (easier exams perhaps, or a cohort effect) such that any change wouldn't be attributable to the CS section. (Never ask a statistician for an unqualified answer!) But what we could do was look at how the gap between CS and non-CS students had changed in terms with a CS section versus terms without.
We found that in terms without CS sections (11130 students), the average difference between CS and non-CS students was 4.2%; whereas in terms with CS sections (4126 students), the average gap closed to 0.9%.
By meeting the students where they are, using examples that are relevant to their interests and future education, and helping them see the applications of the material to their lives and studies, we have improved the performance of CS students and closed the performance gap.
Quite a few students said they were now more interested in pursuing upper year STAT courses than they were before they took STAT 230, and many said they appreciated the effort put in. But by far the most common comment received from students: do the same thing to STAT 231! :)
This post is based on a presentation I made at the 52nd Actuarial Research Conference in Atlanta, Georgia in July 2017. The slides for the presentation are here.
I was inspired to give this talk by a few conversations I had with other actuarial educators at last year's conference. I mentioned my usual approach to testing both basic and high level thinking skills, along with communication, in the same question on my exams. More than one person was extremely surprised by my approach and exclaimed "I wish I could write questions like that!"
I'm here to tell you that you can. No matter what level, what area, what class size, or what background your course is from, there are always ways of incorporating higher level learning into your assessments.
But let's back up a bit. What do I mean by "higher levels of learning"? You may be familiar with Bloom's Taxonomy of Learning, and the work done by Marzano building on it, but if you are not I encourage you to learn more about them.
The key thing to remember is that assessment is curriculum. The way you assess your students is what they will learn to do. So if you want your students to have particular skills (critical evaluation of models, clear communication of ideas, understanding of relationships between concepts, etc), you need to test them on that. You can talk about the importance of (say) good communication skills until you're blue in the face, but if every single question on every assignment and test is calculations only - actions speak louder than words - they will just focus on learning the calculations.
I'm not saying you should give up on testing the basic calculation skills. Those are essential, and should still be tested. But you can incorporate parts of questions that go deeper and allow students to really show you what they know.
Here are some ways I include higher level questions:
When you design a question, you can also do it in a way that makes the marking reasonable. I like to start with a basic calculation, giving the answer to less accuracy so students know they are on the right track, then some more complex calculations, and then a conceptual question extending the material.
There are lots of strategies for incorporating higher level questions into your assessments, and I hope you'll give them a try. The more you do it, the easier it gets to think of questions, and the more you ask this type of question, the more you will encourage your students to learn at a deeper level.
As part of my LITE grant (see posts below) I presented the outcomes of my work with STAT 334 at UW's Opportunities and New Directions conference in April 2016. I also gave a similar (and a bit more detailed) presentation to my own department of Statistics and Actuarial Science in June 2016. Finally, I talked about how to extend these ideas to Actuarial Science courses at the Actuarial Teaching Conference in June 2017 in Pittsburgh.
The slides used for the three presentations were not identical, but all essentially similar to the first one, which is here. If you'd like any more information please feel free to contact me.Part 3: Oral Exams
I have used Oral Exams in several courses in the past (STAT 430/830: Experimental Design, ACTSC 455/855: Advanced Life Insurance Practice, and ACTSC 613: Probability and Statistics for Actuaries) based on research I had done while completing my Certificate in University Teaching. So I was eager to use them in STAT 334 as well. I always use them in addition to, rather than instead of, a traditional written final exam.
If you are interested in further information, I spoke about my use of Oral Exams at the Actuarial Research Conference in August 2015 in Toronto, and my slides from that presentation are here.
Oral Exams have a number of advantages, including:
They are not without concerns however, including nervousness on the part of the students, cheating or bias/subjectivity in grading, and the time it takes to administer. I made several choice in the way I run my Oral Exams, which alleviate most of the concerns.
First is the way the exams are structured. Students are told they will be asked 5 questions (one definition, one advantages/disadvantages, one compare/contrast, one describe a process, and one discuss the impact of a change) and given the opportunity to practice one or two questions, so there are no surprises. The exam is open book so they can bring in absolutely anything (including the textbook, although I recommend making summary notes to be able to find things quickly if they need to look something up). And the exam itself is 15 minutes and only worth 5% of their final mark.
Secondly, in terms of fairness, I have a bank of questions (it is EXTREMELY easy to create questions following the 5 categories above) and pseudo-randomly generate a set for each student so that all the main topics in the course are covered for each person, but no two people have the exact same questions. For the grading, a clear marking scheme helps cut down on subjectivity. For example, it could be that each key point they get without any prompting/help, they get 1 mark, and for each point with prompting they get 0.5 marks, until they have 5 marks or time is up and we move on to the next question. They can also skip or go back whenever they like within the 15 minutes, or ask questions about what I mean.
Finally, the time it takes to run the exams can be prohibitive if you have a large class (I've done it in classes ranging from 18 students up to 60, which was too much. 40 students is 10 hours of exams, for example.) But with a reasonable class size, it can give you a much clearer picture of the knowledge your students have gained, and make them review the material at a much deeper level, which will help them on their written exam too.
I asked some of my former students about their experience with these exams and I'll end with a quote from one of them: "This helped me understand the concepts better when studying for the final exam and in my future courses because it did not just consist of memorizing information and forgetting it immediately after the written exam. I actually had to learn to understand the concepts in order to communicate them effectively."
Part 2: Case Study Competition
A lot of applied statistical techniques are best learned by doing, such as experimental design, predictive modelling, simulation, etc. Probability models (what is covered in STAT 334) are often fairly theoretical, but STAT 334 is a course for business and accounting students, who are more interested in the applications. They are also familiar with the idea of case studies from the business courses they take.
Because of that, I decided to have a Case Study Competition in STAT 334. The choice of topic was completely up to the students (in groups of 3) and they had to take something of interest to their group and model it with a Markov Chain. The competition involved in-class presentations to their peers and a panel of judges, and a written report due after seeing all the presentations, so they could incorporate the judges' feedback and ideas from their fellow classmates. I also had them write a short reflective paper on what they learned from their own project and had them give some critique of 2/3 other groups' projects.
The competition went really well. It was well worth a week of class time to do the presentations, since we saw a huge variety of topics chosen by the students themselves, and many of the course concepts applied in various ways. The judges (other faculty in my department) were very impressed with what the students were able to do with the material after a fairly quick introduction. The funding from the LITE grant mostly went to cash prizes for the competition.
Through the process of choosing the topic, seeking out data sources, creating the transition matrix itself from the data, and then doing the analysis appropriate for their topic, the students gained a much better appreciation for the versatility of the Markov Chain model, as well as recognizing its limitations. Building the model themselves rather than starting from a model handed to them is essential to understanding how it works.
There were too many excellent topics to list them all, but I want to briefly highlight a few:
Overall the students gained a lot of experience and knowledge both from their own project and observing each others' projects. As one of the students wrote in their reflective paper, "The application of Markov Chains in our real life problems allowed us, as a class, to see the very appropriate application of one simple statistical concept in a kaleidoscope of areas in the world we live in today. The experience to me was one of the first, which I felt strengthened my conviction on the application of Math in everyday life."
As a teacher, I can't ask for a better result than that!
Last year I applied for and received a LITE Seed Grant to add some new techniques to STAT 334. The three main things I added were interactive tutorials, a case study competition, and oral exams. I'll talk about each one in a different post.
Part 1: Interactive Tutorial Activities
Different instructors use tutorials in different ways, but for me the most effective way is to give the students essentially an in-class assignment. Open book, they can talk to each other, and the instructor and TA(s) walk around and actively help groups. I've long done this in ACTSC 232 and wanted to try to take it even further in STAT 334.
I had 10 tutorials in the term and varied the activities each week.
The first and last I used for a pre-test and post-test on the course material. That's giving the same test before and after the course so I (and the students!) can see how much they have learned.
The tutorial before the midterm I used as a review activity. I broke the class into 6 groups, and each group was responsible for two things: writing down summary notes of the concepts in a specific few lectures, and answering a midterm-like question on that material. For the second phase we re-shuffled so there were groups of 6 people (one from each of the original groups.) Then in the new groups, each member took turns explaining their summary notes and going through the solution to their question. (I rotated the pieces of paper with that information around the groups manually.) By the end of the class, each person had explained one question and had 5 others explained to them, providing essentially a complete practice midterm and set of study notes, which I then posted on the course website afterwards.
The tutorial after the midterm I used to "take up" the midterm in a non-traditional way. I made groups of 3 such that every group had someone who got each question right or mostly right, gave them their (unmarked) tests back along with a blank test, and had them collaborate on a set of "solutions." Revisiting the material with other people to help made the midterm much more of an active learning experience. If people's answers disagreed, they had to debate who was right. Almost all of the "solutions" were perfect. At the end I gave them marked copies of their own original papers.
The other six tutorials were used to do questions and activities relating to important threshold concepts in the course, which many students have struggled with in the past. These included:
For the Markov Chain activity, in groups of 6, each person represented a "state" (1, 2, 3, 4, or 5) in a Markov Chain and was given a piece of paper with instructions for which state to go to next if a coin came up Heads or Tails. The students traced the path of the chain several times on a piece of paper until they got a sense of how the chain behaved, and then were given a set of questions to answer about the chain. They did this for 3 different chains. Through this activity, I was able to introduce the concepts of classes, communication and accessibility of states, periodicity, transient/recurrent states, and the transition matrix, all without any formal definitions. That helped make all these theoretical definitions more concrete and easy to remember when we then defined them in the next class.
The tutorials were a resounding success. 98% of the students reported in an anonymous post-course survey that they found the tutorials helpful in learning the course material, and they were a lot of fun to run!
I used Lego in a grad level ActSci course and it was great!
I'm teaching ACTSC 635: Actuarial Communications, a small class, and we did an interactive kinesthetic learning activity (similar to Lego Serious Play).
Phase 1: Written communication only (in partners)
Phase 2: verbal communication only (in partners)
Phase 3: communication through a third party (in groups of 3)
Students identified many features of effective communication, such as breaking the instructions into phases, giving names to the pieces, having checkpoints along the way ("now it should look like a T"), using an analogy for the overall shape, etc. The group dynamics in phase 3 developed over time, as the team members began to trust each other and understand their learning styles.
The room buzzed with excitement as students got structures right. In fact, Phase 3 was originally only going to be run once, but they were so eager to fully experience the different perspectives that we extended the activity. The excitement in the room throughout made this a very rewarding experience for all.
To round out the activity, the students will write a short reflection paper on the experience, both to think critically about what they learned and to practice writing. This is a communication course after all!
Diana
Materials (scale as needed for number of students n):