I just wanted to share my comments about the ALIRA Exam; we gave the exam to our Latin 1 and Latin 3 students today.
The exam is very short; the website says it has a 50 minute timer. Most of the Latin 1s were done in under 10 minutes, most of the Latin 3s were done in under 15. I'm a little concerned that there aren't enough tasks before it ends the exam in order to get a true sense of where the student is at. Most of my I2 and I3 scored students saw 12-14 questions, my I4 and 5s saw about 18. Those in the novice range saw fewer than 10.
The Latin sources used were indeed varied in context and in time. I do question the design choice to provide this weird papyrus background with a fake script font for the bulk of the passages. My students said it was distracting and I'd definitely agree. There's really no reason to create a faux-situated experience on an examination in that manner. You can see some examples of it here in the ALIRA sample test.
There was a big show-stopping bug which impacted a handful of students (I did email Language Testing International about it, hopefully I'll hear back soon) and invalidated their scores. I had a couple of solid Latin 3 students score N1 (and one even score BR - below the range). Interestingly enough, all four of them ended on the exact same question (about 13 questions in) and no one else saw that question in their rotation. I hope that the question can be fixed or removed as others take the exam.
In the end, many of my students walked away with a bolstered confidence level and affirmation that they were on the right track. My Latin 3s were, by in large, right where I thought they'd be knowing the students. Most were I2 and I3, a few I4 and 5.
The Latin 1s were the surprise of the bunch; except for 3 N1s (and I suspect a test error here, too), everyone was I1 or above with a large cluster at I3. In essence, ALIRA is telling me that our Latin 1 students are at the same place our Latin 3 students are in terms of ability to comprehend a text. I need some time to digest this data and think about how a few things may be impacting those scores; but my Latin 3s are really the last vestiges of my former life of reliance on a more grammar-translation-reading approach and the Latin 1s, in addition to having a lot more CI-type activities embedded in their daily instruction, also have the added benefit of 2 full years of refinement to Operation LAPIS aiding them as well.
Overall, though, I'm very pleased with the ALIRA exam, the time to administer it, and the information (if it is indeed accurate) that it provides. I'm not convinced it's a $10 test, however. That price needs to come down so that I (and others) can administer it program-wide each year.
In an effort for those of us using ALIRA to gauge where our students are at as a whole, I want to start compiling data for exams given this year. I created this Google Form for anyone who wants to contribute:
https://docs.google.com/forms/d/1sx87t04SkDYByBWXKghbmXJHm5iAq3Xjc7Nuf_uvSVo/viewform
Names and schools won't be published (included that space, though, to help ensure that we're not getting false data), only raw numbers for proficiency at each course level.
If you have had experience with ALIRA, it would be great to share your thoughts, so please do!
Showing posts with label assessment. Show all posts
Showing posts with label assessment. Show all posts
4.09.2014
1.05.2013
On Narrative, Collection, and Re-engagement
Traditionally, this a tough time of year for a certain cross section of my Latin I classes.
Traditionally, this is the point where the complexity of the Latin advances to the point where those skating by, not really understanding what a direct object is, or why it has a different set of endings than a subject, are now underwater.
Traditionally, this is where the extra noun declension sheets, or verb conjugation sheets, run hot off the copier and into their hands in a frantic attempt to prepare for the midterm exam.
Traditionally, this is a point when a small portion, no matter what kind of interventions are in place, fall off completely and check out for the rest of the year.
Tradition isn’t tradition any longer.
One of the powerful affordances of a continuous narrative -- a narrative in which the students have an active role in shaping on a daily basis -- is the opportunity to re-engage at almost every moment. They aren’t reading about some figures whom they have no attachment to but rather they are playing their own characters, characters whom they have a genuine investment in and ones that they genuinely care about.
One of the powerful affordances of working in a permanent social collaborative unit is that on any given night their team is ready to help them get back into the fold and contribute in a meaningful way -- even if they haven’t contributed for the last two weeks. Rather than competing against their classmates, in its place there is a common sense of camaraderie and companionship on their journey of learning Latin and subsequently saving civilization.
One of the powerful affordance of the collection mechanic is that it provides a simple, effective, and straightforward way to focus on small bits of grammar at a time in a way where progress is marked visually not just in their lists but also in their acclimation of Latinity Points for taking part in the grind.
One of the powerful affordances of an easy re-entry is that the student who may have been struggling now has a reason and a way to get back into the game and, more importantly, a way to demonstrate learning and growth that’s real and tangible to them because they can see the immediate effects of their renewed participation and engagement.
One of the powerful affordances of tossing out the old traditions is that you get to rewrite the rules.
Consider the rules rewritten.
Traditionally, this is the point where the complexity of the Latin advances to the point where those skating by, not really understanding what a direct object is, or why it has a different set of endings than a subject, are now underwater.
Traditionally, this is where the extra noun declension sheets, or verb conjugation sheets, run hot off the copier and into their hands in a frantic attempt to prepare for the midterm exam.
Traditionally, this is a point when a small portion, no matter what kind of interventions are in place, fall off completely and check out for the rest of the year.
Tradition isn’t tradition any longer.
One of the powerful affordances of a continuous narrative -- a narrative in which the students have an active role in shaping on a daily basis -- is the opportunity to re-engage at almost every moment. They aren’t reading about some figures whom they have no attachment to but rather they are playing their own characters, characters whom they have a genuine investment in and ones that they genuinely care about.
One of the powerful affordances of working in a permanent social collaborative unit is that on any given night their team is ready to help them get back into the fold and contribute in a meaningful way -- even if they haven’t contributed for the last two weeks. Rather than competing against their classmates, in its place there is a common sense of camaraderie and companionship on their journey of learning Latin and subsequently saving civilization.
One of the powerful affordance of the collection mechanic is that it provides a simple, effective, and straightforward way to focus on small bits of grammar at a time in a way where progress is marked visually not just in their lists but also in their acclimation of Latinity Points for taking part in the grind.
One of the powerful affordances of an easy re-entry is that the student who may have been struggling now has a reason and a way to get back into the game and, more importantly, a way to demonstrate learning and growth that’s real and tangible to them because they can see the immediate effects of their renewed participation and engagement.
One of the powerful affordances of tossing out the old traditions is that you get to rewrite the rules.
Consider the rules rewritten.
5.13.2012
Latinity Points (XP) as a Replacement for Grades, Part 4
In the previous three parts of this exercise, I laid out the criteria required in order to evaluate without grading, explained how Latinity Points are utilized as a part of Operation LAPIS, and then provided a full discussion and response from a group of three students. In this final part my aim is to elaborate on what the end of that process looks like, specifically when it comes time to assign a traditional grade in a very non-traditional classroom. As a reminder from the first post, the following were the four criteria that I argued had to be a part of this system.
1) Continuous embedded formative assessment: Every component of Operation LAPIS is designed with this at the core. There are no superfluous mechanics or deliberately gamified elements for cheap extrinsic motivation. Every episode requires the students-as-operatives not only to read and comprehend Latin but also to apply relevant cultural and historical information through both guided and independent research. Every episode also requires them to collaborate with their teammates to roleplay their character’s actions through a particular world-view. As built in differentiation, students can be challenged to build their responses in original Latin composition as well -- which many of them attempt at least in some small part. Simply put, every element requires them to actively draw on skills they’ve gained while applying new skills and knowledge.
2) A record system with meaningful feedback: All of their LPs are awarded through a shared (between myself and the student) document known as the Operative Dossier. Once we start to get into a grove at the start of the year, the students understand that an LP award in the 260s or higher is on the right track, while something around 200s means that more work is needed. Included in each dossier is a ‘Transmission from Mission Control’ which can provide direct feedback in a way that doesn’t break from the immersion of the ‘students-as-operatives’. Since all operatives have access to the rubric, specific attention can be drawn to parts of their contributions that did not meet expectations.
3) A record of all student work: This is the easy part, thanks to Google Docs and Edmodo as the platform of choice for collaboration and discussion. All student work in Google Docs is shared between myself and the student. At any point, through the commenting feature in Google Docs, I’m able to provide specific and detailed feedback. Since LPs are deliberately not a masked grading system, operatives can choose to respond to the feedback, fix errors, and gain more LPs for revising their work.
All team discussion happens in secure sub-groups on the Edmodo platform. If there is every a question from a parent (or from a student), it takes almost no time to call up a record of the student’s contributions to reflect on how they are (or aren’t) meeting the criteria spelled out on the rubric. This record of work is also important as teams will often recall information from the past immersions as being useful for the current -- scrolling down their team discussions quickly gives them access to that material in a way that a test or quiz shoved into a binder, or the nebulous black hole known as a backpack, simply cannot.
Now for the fun part.
4) Student agency in the evaluation process: When it comes time to report grades at the prescribed four times per year, I ask all of my students to first submit a self-reflection piece. I ask them to evaluate their strengths, their weaknesses, mention one or two episodes that they felt they did a fantastic job contributing to, and also to mention one or two episodes where they think they could have provided more to the team discussion. In addition, we take a look at the current LP totals and I give them an intentionally wide range of values for letter grades (intentional overlap, as well). Ideally, I’d sit down with each student individually and have a face-to-face discussion, but with 25+ students in each classroom, logistically it would be too difficult to accomplish as a part of a standard high school schedule. Instead I require the students to email me their response, which, in of itself, hits on some of the secondary learning objectives of written communication, etc. At the end of their self-reflection, they assign themselves a grade which they think they are deserving of. The wonderful thing about this endeavor is that almost always they hold themselves to tougher standards than I and, in my reply, I get to “talk them up” a bit.
Do I get the occasional student that says, “Sweet? I get to assign myself a grade? I get an A!”? Absolutely. However, with the continuous embedded formative assessment, feedback through carefully constructed mechanic, and an accessible record system, it then because a teachable moment when they are asked to demonstrate in what ways they met the learning objectives to warrant that ‘A’.
- Continuous embedded formative assessment of progress
- A record system with meaningful feedback towards meeting learning objectives
- A record of all student work
- Student agency in the evaluation process
1) Continuous embedded formative assessment: Every component of Operation LAPIS is designed with this at the core. There are no superfluous mechanics or deliberately gamified elements for cheap extrinsic motivation. Every episode requires the students-as-operatives not only to read and comprehend Latin but also to apply relevant cultural and historical information through both guided and independent research. Every episode also requires them to collaborate with their teammates to roleplay their character’s actions through a particular world-view. As built in differentiation, students can be challenged to build their responses in original Latin composition as well -- which many of them attempt at least in some small part. Simply put, every element requires them to actively draw on skills they’ve gained while applying new skills and knowledge.
2) A record system with meaningful feedback: All of their LPs are awarded through a shared (between myself and the student) document known as the Operative Dossier. Once we start to get into a grove at the start of the year, the students understand that an LP award in the 260s or higher is on the right track, while something around 200s means that more work is needed. Included in each dossier is a ‘Transmission from Mission Control’ which can provide direct feedback in a way that doesn’t break from the immersion of the ‘students-as-operatives’. Since all operatives have access to the rubric, specific attention can be drawn to parts of their contributions that did not meet expectations.
3) A record of all student work: This is the easy part, thanks to Google Docs and Edmodo as the platform of choice for collaboration and discussion. All student work in Google Docs is shared between myself and the student. At any point, through the commenting feature in Google Docs, I’m able to provide specific and detailed feedback. Since LPs are deliberately not a masked grading system, operatives can choose to respond to the feedback, fix errors, and gain more LPs for revising their work.
All team discussion happens in secure sub-groups on the Edmodo platform. If there is every a question from a parent (or from a student), it takes almost no time to call up a record of the student’s contributions to reflect on how they are (or aren’t) meeting the criteria spelled out on the rubric. This record of work is also important as teams will often recall information from the past immersions as being useful for the current -- scrolling down their team discussions quickly gives them access to that material in a way that a test or quiz shoved into a binder, or the nebulous black hole known as a backpack, simply cannot.
Now for the fun part.
4) Student agency in the evaluation process: When it comes time to report grades at the prescribed four times per year, I ask all of my students to first submit a self-reflection piece. I ask them to evaluate their strengths, their weaknesses, mention one or two episodes that they felt they did a fantastic job contributing to, and also to mention one or two episodes where they think they could have provided more to the team discussion. In addition, we take a look at the current LP totals and I give them an intentionally wide range of values for letter grades (intentional overlap, as well). Ideally, I’d sit down with each student individually and have a face-to-face discussion, but with 25+ students in each classroom, logistically it would be too difficult to accomplish as a part of a standard high school schedule. Instead I require the students to email me their response, which, in of itself, hits on some of the secondary learning objectives of written communication, etc. At the end of their self-reflection, they assign themselves a grade which they think they are deserving of. The wonderful thing about this endeavor is that almost always they hold themselves to tougher standards than I and, in my reply, I get to “talk them up” a bit.
Do I get the occasional student that says, “Sweet? I get to assign myself a grade? I get an A!”? Absolutely. However, with the continuous embedded formative assessment, feedback through carefully constructed mechanic, and an accessible record system, it then because a teachable moment when they are asked to demonstrate in what ways they met the learning objectives to warrant that ‘A’.
4.24.2012
Latinity Points (XP) as a Replacement for Grades, Part 3
Continuing from Part 2, here is a complete transcript from one of the teams engaged in a recent mission.
The quick synopsis: the Recentiī were thrust back into a historical training mission. This time, they were on the deck of Duilius' flagship just prior to the important Battle of Mylae during the First Punic War. Remember, Rome had just lost a number (read: every) major naval battle since the outbreak of war, including most recently at the Lipari Islands where Scipio Asina was captured by the Carthaginian fleet.
Duilius, feeling a bit uneasy (and rightfully so), required a little pep talk despite the tactical advantage he was about the unveil -- the corvus. At the moment the Recentiī encounter him, Duilius is questioning the advice given to him by the Corneliī and the Claudiī about the nature of the patrician class, the plebeians, and how to manipulate power in Rome. My commentary will follow the full transcript:
Student 1- so i was thinking that we need to find another was to glorify Rome without tricking the plebs...wait -_- thats what the paragraph says..
Student 1 - hmmm it turns out that the same fate that happened to Appius Caudex and Scipio Asinus happened to Duilius...he was given control during the war, but after that he "never held another office of high command" i wonder why? i mean he seems like a nice enough guy...not one to be a "donkey".
Student 2 - Maybe in these men's cases, the plebs caught on to what the two patricians had done and scorned them for the rest of their political careers choosing instead to support other figures?
Student 2 - Oooh, I bet Student 3 and I can relate this to some recent thing we learned in History class. Machiavelli, a Renaissance writer, produced The Prince which highlighted what he thought were the necessary qualities to a successful leader. It included things as being deceptive, any immoral means as long as it accomplishes the desired end results. That's what Cornelii and Claudii had advocated and currently, Asinus is trying to convince Dulius to do.
Student 2 - So correlating to our need to suggest some guidelines which Dulius should follow in order to achieve victory and fame, we can give him tips which he should not do.
Such as:
1.) Being impatient ( Eager to secure such an important port and to cover himself with glory, Scipio rushed to the Lipari islands without considering security.)
2.) Rallying up his men, which luckily enough, Dulius is planned on doing though he has no idea how... (The crews panicked and escaped to land, leaving the ships unattended and Scipio Asina to be made prisoner by the Carthaginians).
Student 2 - Oh, I didn't know who you were referring to when you said he never held another office of high command. Disregard my post below yours. Or.. I can just delete it. :D
Student 3 - Exactly Student 2! That's what I was thinking. They thought it was better to be feared than loved. Maybe we should just tell Dulius to do what he knows is considered to be "lofty". (5 points for english vocab :D)
Student 2 - @Student 1 That is a good point though, it's ironic. "Neither the humiliation, nor that Scipio Asina was the first Roman admiral to lose a battle, ended his career; in 254 BC, Scipio Asina was elected consul for the second time"
Student 3 - It's almost as if the defeat ruined his career......sounds like they don't like failure....
Student 2 - Maybe to preserve the kind of honor and respect he received (which does come by every day), Dulius secured his position by eliminating any possibility of him being humiliated by loss or failure.
Student 2 - Or as part of his plan to be "good cop" and not "bad cop" like Asinus wanted, he just played modest.
Student 3 - We could also tell him not to expect too much but just treat it as a normal battle. Stress often brings effort and quality down....
Student 2 - What gens is Dulius? Will that offer us to some insight as to what he was brought up to support?
Student 1 - I need to correct my last statement...i wasnt talking about Scipio Asinus or Appius Caudex...I mean that Duilius suffered the same fate as them
Student 2 - "Not much is known about his family background or early career, since he was a novus homo, meaning not belonging to a traditional family of Roman aristocrats".
Student 1 - so at least he raised his own rank..he relates to the plebs
Student 3 - Ok, so basically we're just telling him good luck lol.
Student 2 - That means Dulius is not naturally inclined to exploit the plebs or Roman population as easily as the patricians would've. Why should we mold him in their face? We should let him be, it seems as though that's what he had done in historical context, be honest with his fleet.
Student 2 - After all, he did win the title of Censor. "The censor was an officer in ancient Rome who was responsible for maintaining the census, supervising public morality, and overseeing certain aspects of the government's finances."
Student 2 - Emphasis on PUBLIC MORALITY.
Student 3 -Oh wow then good. Looks like we got a plan :D
Student 2 - So are we agreed on this, Dulius should abandon and trash all of the notions, talking of Patricians and influencing the population selfishly, delivered to him by Asinus?
So what was the end result? After their discussion on Edmodo, the crew headed over to Google Docs and collaboratively hashed out their response for the evening. Looking back on the revision history, it is easy to see that it was a fairly balanced effort by all three members, especially the Latin composition passage. I present it, unedited, below:
Horatiana pats the back of a troubled friend in need, linked by mutual suffering. She too knew only too well the stress and pressure of being expected to live up to something practically unattainable. Left unattended, this problem can be crushing when the expectation is not met or properly challenged; casting a lifelong burden. Especially when one more loss could spell out the untimely end of the Roman civilization, the pressure is unbearable upon the shoulders of one man. She wished with her heart and soul to dispel the fear radiating from this man.
Horatiana ad vultum Duilii spectat. ille sudabat et tremebat valde! illa debet cogitare celeriter aut omnis spes est perdita. scivit Appium Caudem et Scipium Asinum luserant populum. Et illi facerunt id solummodo sibi. Duilius non debet agere illud si vult populum amare eum. Est optimum acquiere fides classum. Bonus imperator, sicut tu, non requirat potentiam Lapidis. Scipio habuit lapidem et ille erat captivus. Classes faciunt imperatorum potentiam.
Horatiana, in the last fleeting minutes before approaching the islands, drills Dulius in the makings of a fine leader. The patience, the strategy, bravery, modesty, listening skills, the willingness to sacrifice oneself for the bigger picture, and most of all, to act as an extension of the navy itself, amplifying its strengths many times. Speak from your heart in all honesty to your men. You do not need deception to get them to fall in line. Fear only works to an extent. No, what you need is compassion. If you speak with true decency, then why will there be any doubt the men will not show the same love you have for Rome? You need not to worry about your men, but instead how you can guide them with virtue because only then will they live up to your standards. Or slave away as one of the patricians' mindless puppets. That's what the Claudii and Cornelii probably had in store for you. But deep down, you had always doubted their methods. You're a novus homo, for goodness' sake.
Already I'm crossing over to TL:DR territory so I'll keep my commentary short. We have a standard response rubric located at this link. I'll focus on Student 2 for assessing how well they did meeting the learning objectives.
Remember, this was a single night's worth of work and they received no grade on it in the traditional sense. Student 2 receive 292 LPs for their work on this immersion prompt. Contributions such as Student 2's are the easiest the award points for -- I think most readers will agree this student exceeded standards for a Latin I class for Expression of Ideas, Elaboration, Cultural Competence, Risk Taking, and also the two Latin composition categories, Grammar and Vocabulary. I especially liked this student's ability to connect to what they were learning in their history class about Machiavelli and his views on how a leader should operate. Student 2 was respectful and helpful towards the other team members and it is clear that all members benefited greatly from the social and collaborative setting.
The collaborative setting also allows an opportunity for different members to showcase their talents in different ways -- Student 2, for example, excels at research and pulling out relevant information and distilling that for the team. Another student has a knack for elaborating the role-playing for their Recentia, Horatiana, and the third does very well with fine tuning the grammar in their compositions.
Again, I want to stress that all of this work (and it is substantial) is done with almost no expectation of extrinsic rewards. These students (and many of their classmates) are taking part in exploring these missions at great lengths purely for the learning that comes out of doing so. In the final installment, I'll spend some time reflecting about how I leverage Latinity Points as a launching point for a discussion about evaluation -- not the basis for evaluation -- when it comes time to report formal grades for marking periods.
4.02.2012
Latinity Points (XP) as a Replacement for Grades, part 2
Continued from part 1...
Why Latinity Points as an experience point (XP) system? For those unfamiliar with what experience points are, a quick trip to Wikipedia yields:
‘Quantify progression’ are the vital words of that definition. Experience points are a visual representation of progression towards an objective. In the classroom, the objective should be mastering the learning objectives for that course. In Operation LAPIS, Latinity Points (LPs) serve to quantify an operative’s progression towards the overarching learning objectives; read, think, act, write, and speak like a Roman.
What Latinity Points are not:
Before I go too much further, I’d like to share some thoughts on how I think XP systems are misused and abused in the classroom. XP should not be window dressing for standard grading mechanisms. If all you are doing is announcing that a particular assignment is worth 100 XP, and receiving 90 XP on the assignment is an A, then you are simply putting lipstick on a pig. That’s cheap gamification and you’ll quickly find that the novelty wears off. If you set a cap for the marking period of, say, 10,000 XP and then convert experience point totals to a grade with a formula, you’ll discover the same problem. The end result becomes amassing XP for the sake of the letter grade at the end, rather than progression towards the learning objectives. The end result becomes finding the path of least resistance in order to achieve that goal -- the same problem plaguing us with current traditional grading systems. Our students do not want to take risks, and will not take risks, because they are punished for doing so. An XP system grafted onto a standard grading scheme does not resolve that problem.
What Latinity Points are:
LPs are quantitative representations of a student's progression through the course. Like rewards for completion of quests in a game, they can be awarded for any task inside of the operation: team collaboration, attunement challenges, successful completion of elements in the collection mechanic, etc. Since all activities in Operation LAPIS, like any good game, are continuous embedded formative assessments definitively tied to learning objectives, the LPs gained from any one activity serve to mark overall advancement.
LPs are separated from a grade value; the better you do a task, the more Latinity Points you receive. They are capped per activity but the same holds true in almost all games. You can only grind so many points of out slaying level 5 goblins before you need to move on to bigger challenges. This encourages students to take risks they wouldn’t otherwise take since there is no potential for “loss” of points.
LPs are paired with meaningful feedback rather than numbers divorced from the activity. In the example of team collaboration in immersion episodes, the LPs awarded are tied to a standards-based rubric. In addition, the affordances of a Google spreadsheet as an ‘Operative Dossier’ allows for easy personalized feedback in the form of in-cell comments to help shape better contributions from the student in future episodes.
LPs are the beginning of the conversation with a student when it comes time to report a marking period grade. Because they are tied to progression, LPs (and XP systems in general) serve a powerful function to allow the student to tell the narrative of their learning for that particular period in a way that traditional grading systems just don’t allow because of the inherent connotations that grades hold. Instead, the student becomes an agent of true self-assessment and, for many, a much harder critic than I would have been.
In the next part, I’ll break down examples of how and why I award LPs and what those mean for the overall picture of evaluating student learning.
Why Latinity Points as an experience point (XP) system? For those unfamiliar with what experience points are, a quick trip to Wikipedia yields:
An experience point (often abbreviated to Exp or XP) is a unit of measurement used in many role-playing games (RPGs) and role-playing video games to quantify a player character's progression through the game. Experience points are generally awarded for the completion of quests, overcoming obstacles and opponents, and for successful role-playing.
‘Quantify progression’ are the vital words of that definition. Experience points are a visual representation of progression towards an objective. In the classroom, the objective should be mastering the learning objectives for that course. In Operation LAPIS, Latinity Points (LPs) serve to quantify an operative’s progression towards the overarching learning objectives; read, think, act, write, and speak like a Roman.
What Latinity Points are not:
Before I go too much further, I’d like to share some thoughts on how I think XP systems are misused and abused in the classroom. XP should not be window dressing for standard grading mechanisms. If all you are doing is announcing that a particular assignment is worth 100 XP, and receiving 90 XP on the assignment is an A, then you are simply putting lipstick on a pig. That’s cheap gamification and you’ll quickly find that the novelty wears off. If you set a cap for the marking period of, say, 10,000 XP and then convert experience point totals to a grade with a formula, you’ll discover the same problem. The end result becomes amassing XP for the sake of the letter grade at the end, rather than progression towards the learning objectives. The end result becomes finding the path of least resistance in order to achieve that goal -- the same problem plaguing us with current traditional grading systems. Our students do not want to take risks, and will not take risks, because they are punished for doing so. An XP system grafted onto a standard grading scheme does not resolve that problem.
What Latinity Points are:
LPs are quantitative representations of a student's progression through the course. Like rewards for completion of quests in a game, they can be awarded for any task inside of the operation: team collaboration, attunement challenges, successful completion of elements in the collection mechanic, etc. Since all activities in Operation LAPIS, like any good game, are continuous embedded formative assessments definitively tied to learning objectives, the LPs gained from any one activity serve to mark overall advancement.
LPs are separated from a grade value; the better you do a task, the more Latinity Points you receive. They are capped per activity but the same holds true in almost all games. You can only grind so many points of out slaying level 5 goblins before you need to move on to bigger challenges. This encourages students to take risks they wouldn’t otherwise take since there is no potential for “loss” of points.
LPs are paired with meaningful feedback rather than numbers divorced from the activity. In the example of team collaboration in immersion episodes, the LPs awarded are tied to a standards-based rubric. In addition, the affordances of a Google spreadsheet as an ‘Operative Dossier’ allows for easy personalized feedback in the form of in-cell comments to help shape better contributions from the student in future episodes.
LPs are the beginning of the conversation with a student when it comes time to report a marking period grade. Because they are tied to progression, LPs (and XP systems in general) serve a powerful function to allow the student to tell the narrative of their learning for that particular period in a way that traditional grading systems just don’t allow because of the inherent connotations that grades hold. Instead, the student becomes an agent of true self-assessment and, for many, a much harder critic than I would have been.
In the next part, I’ll break down examples of how and why I award LPs and what those mean for the overall picture of evaluating student learning.
4.01.2012
Latinity Points (XP) as a Replacement for Grades, part 1
For many of us it is time for that ritual again -- namely, posting quarterly grades for our students. However, because of this fact it is also a good time to reflect on grading practices and just who those grades are for. Since we’ve opened up the free two mission trial of Operation LAPIS, it is also a good time to explore alternatives to the traditional grading system; Latinity Points (XP as a measure of advancement towards learning objectives.) Bear with me because this will be spread out over a few posts.
Grading, as it is currently conceived, is nothing more than a purely extrinsic motivator. In order to move forward we, as educators, have to stop kidding ourselves that “grading” student work is about providing feedback. We’d like it to be about the feedback but it isn’t. It’s about that mark at the top, whether it be a number or a letter. When you hand back a quiz, a paper, or any other assignment, take a look at your students’ reactions and behavior. I’d wager it follows the same pattern each time:
If that is what you see then grading is not about the feedback. It is about the extrinsic reward for completing a particular task to your specifications. It is about instant gratification, regardless of how that outcome was achieved. It is also about the competition with peers. It’s why, a week prior to grades closing, your students inevitably ask the question, “What can I do for extra credit?” or immediately following the student receiving the report card, “Why did you give me a B?” It’s not about the learning and it hasn’t been for a long time.
But wait -- many educators will be quick to stop me in my tracks and ask, “How do you provide evidence and justification for assigning grades if you aren’t ‘grading’?” A valid question, sure, because of the perceived value that “accountability” holds in the culture of school -- as if “accountability” can exist only as numbers in a gradebook. If not grades, then how can you evaluate a student’s progress towards learning objectives? For starters, I’m working with the students day in and day out. I’m observing their progress in and out of the classroom. But if my professional judgement isn't enough, I’ll argue that four criteria need to be met:
In the next installment, I’ll discuss how the Latinity Point mechanic of Operation LAPIS hits all four of those criteria to provide an alternative to standard grades.
Grading, as it is currently conceived, is nothing more than a purely extrinsic motivator. In order to move forward we, as educators, have to stop kidding ourselves that “grading” student work is about providing feedback. We’d like it to be about the feedback but it isn’t. It’s about that mark at the top, whether it be a number or a letter. When you hand back a quiz, a paper, or any other assignment, take a look at your students’ reactions and behavior. I’d wager it follows the same pattern each time:
- Student goes straight for the number or letter, skipping any corrections or written feedback.
- Student looks to friend, neighbor, or person sitting near them and asks, “What’d you get?”
If that is what you see then grading is not about the feedback. It is about the extrinsic reward for completing a particular task to your specifications. It is about instant gratification, regardless of how that outcome was achieved. It is also about the competition with peers. It’s why, a week prior to grades closing, your students inevitably ask the question, “What can I do for extra credit?” or immediately following the student receiving the report card, “Why did you give me a B?” It’s not about the learning and it hasn’t been for a long time.
But wait -- many educators will be quick to stop me in my tracks and ask, “How do you provide evidence and justification for assigning grades if you aren’t ‘grading’?” A valid question, sure, because of the perceived value that “accountability” holds in the culture of school -- as if “accountability” can exist only as numbers in a gradebook. If not grades, then how can you evaluate a student’s progress towards learning objectives? For starters, I’m working with the students day in and day out. I’m observing their progress in and out of the classroom. But if my professional judgement isn't enough, I’ll argue that four criteria need to be met:
- Continuous embedded formative assessment of progress
- A record system with meaningful feedback towards meeting learning objectives
- A record of all student work
- Student agency in the evaluation process
In the next installment, I’ll discuss how the Latinity Point mechanic of Operation LAPIS hits all four of those criteria to provide an alternative to standard grades.
6.28.2011
Reflections, Part 3
Back in October, I posted about a series of events that lead to a drastic shift in my grading and homework philiosophy in my Latin II classroom. You can read about the backstory if you’d like, but in sum a single comment “Latin kills me inside” prompted me to try out the ideas that educational reformers like Alfie Kohn have been pushing for years.
Simply put, I was removing all traditional “grading” from my classroom and instead putting in its place a self-evaluation “masteries chart” of all of the skills we expect our Latin II students to have at the end of the year. In addition to that chart, I also established time at each marking period to have one on one conferences with the students to talk about their accomplishments, areas that they struggled in, and what they were going to work on in the next quarter.
Does this mean I stopped giving quizzes and tests? Absolutely not. Instead, after we took those assessment checks, we went over them in detail and the students made their own corrections to see and understand their mistakes. Occasionally I collected their work to make sure that they were doing the self-corrections but eventually I found that I just didn’t need to do that: they were doing their corrections and doing them quite well. In comparison to their Latin II peers with my other colleagues, all of our sections performed nearly identical on the National Latin Exam. There was no degradation in ability or skill as a result of removing grades from the picture.
As the second semester went by, I slowly weened them (or myself?) off of homework. Because of larger systemic issues with homework, a very small percentage of students were actually doing the work that I assigned. Instead, a lot of the exercises and tasks that I would have assigned at home, we worked on in class either individually, in small groups, or as a large class group. Again, the result here was that 100% of the students got the extra reinforcement of the language skills rather than 20-40%. Did this take away from class time that would otherwise be used for other things? Absolutely -- it took away from the time that I normally used for direct instruction. Instead, that time was limited to only when introducing new topics, another huge departure from business as usual.
Did these changes impact how much material we covered during the course of the year? It didn’t at all. In fact, I’d argue that at the end of this year my students as a whole had a better understanding and recoginition of the advanced clauses than any of my past groups. I’ve seen students who, in the past, might have been treading water through Latin II, sneak by in III, accomplish their language requirement and bail out now approach reading and studying Latin with renewed interest and vigor because they are confident in their ability. With the threat and/or reward of grades removed, by in large the students instead focused on learning Latin for the sake of learning Latin. This resulted in a far more enjoyable experience for both my students and myself.
I should also add that before I even started this transition, I approached both my department head and building principal to let them know exactly what I was planning, why I was doing it, and what steps I would be doing to ensure that the students are demonstrating mastery of the material. They were fully supportive with the understanding that, like the initial launch of Operation LAPIS, if things were to go terribly wrong, I’d pull the plug and readjust.
So how about the students? To quote LeVar Burton, “You don’t have to take my word for it...” Here is just a small sample of the feedback provided by my Latin II students at the end of the year, specifically about the shift in grading philosophy:
Wouldn’t it be great if these were the thoughts of every student at the end of every course?
Simply put, I was removing all traditional “grading” from my classroom and instead putting in its place a self-evaluation “masteries chart” of all of the skills we expect our Latin II students to have at the end of the year. In addition to that chart, I also established time at each marking period to have one on one conferences with the students to talk about their accomplishments, areas that they struggled in, and what they were going to work on in the next quarter.
As the second semester went by, I slowly weened them (or myself?) off of homework. Because of larger systemic issues with homework, a very small percentage of students were actually doing the work that I assigned. Instead, a lot of the exercises and tasks that I would have assigned at home, we worked on in class either individually, in small groups, or as a large class group. Again, the result here was that 100% of the students got the extra reinforcement of the language skills rather than 20-40%. Did this take away from class time that would otherwise be used for other things? Absolutely -- it took away from the time that I normally used for direct instruction. Instead, that time was limited to only when introducing new topics, another huge departure from business as usual.
Did these changes impact how much material we covered during the course of the year? It didn’t at all. In fact, I’d argue that at the end of this year my students as a whole had a better understanding and recoginition of the advanced clauses than any of my past groups. I’ve seen students who, in the past, might have been treading water through Latin II, sneak by in III, accomplish their language requirement and bail out now approach reading and studying Latin with renewed interest and vigor because they are confident in their ability. With the threat and/or reward of grades removed, by in large the students instead focused on learning Latin for the sake of learning Latin. This resulted in a far more enjoyable experience for both my students and myself.
I should also add that before I even started this transition, I approached both my department head and building principal to let them know exactly what I was planning, why I was doing it, and what steps I would be doing to ensure that the students are demonstrating mastery of the material. They were fully supportive with the understanding that, like the initial launch of Operation LAPIS, if things were to go terribly wrong, I’d pull the plug and readjust.
So how about the students? To quote LeVar Burton, “You don’t have to take my word for it...” Here is just a small sample of the feedback provided by my Latin II students at the end of the year, specifically about the shift in grading philosophy:
“I love how we weren’t graded this year, it made the class a lot less stressful and allowed me to just focus on learning, rather than worrying about grades.”
“The new style of teaching (and learning) you implemented really worked for me when I thought it would fail. I didn’t know how you could learn without taking tests or quizzes to get a number grade but I realized that you could get a lot more out of the class by focusing on things other than the grade.”
“I used to suck at Latin and now I suck less.. actually, I don’t suck at all. It just now dawned on me how much we learned this year and I didn’t even realize it because I didn’t obsess over a grade at the top of the page.”
“I feel like it was a lot easier to focus on learning the grammar parts when we didn’t have to worry about quizzes. I also think that the elemination of actual “grading” really encouraged learning and made things our own choice instead of doing things just for the A”
Wouldn’t it be great if these were the thoughts of every student at the end of every course?
6.10.2011
Reflections, Part 1
The first full year run of Operation LAPIS is in the books. The final mission was finished on Monday. The exam was completed yesterday. I've learned a great deal over the past academic year and in a series of posts, I plan to reflect upon and share some of the anecdotal pieces of evidence on just how powerful practomimetic learning can be. Next week we'll be traveling to Madison, Wisconsin to the Games, Learning and Society Conference and I'm very excited to share LAPIS with a wider game-based learning audience.
So where to begin?Originally I started the year teaching one section of Latin I that was presented in the traditional format while the other section came in on the first day to find out they were tasked with saving western civilization as we know it. As the year went on and the other section caught wind of what their peers were up to, eventually I had to cave and in the middle of March, I brought the other group into the fold; albeit on the fast track through the missions and the storyline. This proved to be a fortunate decision because the newer group proved to me something that I had long thought would be true -- the full narrative is indeed vital for thorough and meaningful fulfillment of the learning objectives.
While the newer group was clearly impacted positively in terms of engagement with the material, because they were on such a fast track, they just didn't have the deeper understand or connection with the material as the original full-year LAPIS group. This became most evident on their prompt responses for the final mission. After reading and translating a passage of decent length, the students (as part of their final assessment) had to do what they've done all year: respond to the immersion prompt, taking into account the world-view of their character. The only difference was this one was a solo mission.
Over the past few weeks, as I noted in previous posts, they were researching and giving speechs as part of the trials to gain access to the headquarters of the Societās Potentium. All of those speeches were intentionally laying the foundation for them to understand the collapse of the Roman Republic and the establishment of the Principate under Augustus. They did a great job at understanding how the Gracchi led to Marius which lead to Caesar... but could they tie it all together for Antony and Octavian?
The answer was that in a timed exam, under pressure, the results were very mixed. But at the very least they could demonstrate their awareness of how Octavian fit into the big picture. However, I do want to show an example of one of the best responses from the newer LAPIS group (the third of the year group) and the original full year group.
Example from the newer group:
If I could align myself with either Octavian or Anthony I would align with neither of them. Octavian although a great leader, used his power to persuade the Roman senators into a fight that didn’t nessasarily need to occur. When he wanted to start the war, he made sure to persuade the senators to declare war, so it was official and couldn’t be argued against. He started the conflict that would end the lives of both Anthony and Cleopatra. Because my Recentius’s view is very political, Octavian persuading the senators to start this war seems very harsh and it almost seems as though he wanted a war for no reason, but turned the senators against Anthony. Now Anthony on the other hand isn’t perfect either. He left Rome to be with Cleopatra and told her he would give her land that the Romans owned. This is completely going against the idea that he wanted to help Rome at all. I think that him making a decision like this was a good reason for the Romans to be upset but maybe not so upset that a war turned to the outcome, because the land was still technically owned by the Romans. It wasn’t Anthonys to be giving away. So in the end I believe that both sides had mistakes that they seemed to make and in the end the both chose the wrong decision to stick with. They both could have thought their decisions through before they made them.
This is a great, thoughtful, and thorough response to the question of whom would they align themselves with. This particular student was the only one who chose to aid neither man. I love the risk taking that happened here that I'm not sure would happen in any other setting.
Now, for those wondering what true practomimetic learning looks like, here's something that I would love to make into a poster:
Titus thought for a moment on what would be the correct choice, tapping his chin in contemplation of the matter while he considered the story that Sextus had just relayed to him. It certainly did clear some things up, that much was true. With a heavy sigh, Titus turned his eyes from the ground up to the image of Sextus and declared. “I align myself with Octavius.” he stopped to collect his thoughts and became just slightly distracted by the annoying sting of the arm wound he acquired earlier in the day, a bit sickened by the fact that he had only earlier engaged in mortal combat and was now supposed to work up the mind to make a firm decision. Ah, saving the world is quite a fair burden, eh? Shaking his head of further complaint, he cleared his throat and continued “Having been the true heir of Caesar, it was only natural that the late Augustus seized power. ‘Tis the goal of all men who have ambition, it seems. And the conflict with Antony could be seen as inevitable, since Rome has been through the same sad song-and-dance time and time again. The Second Triumvirate’s rather rocky relations made it far too unwieldy to properly oversee the internal functions of Rome, so in some way it was best for the three, that is, Octavius, Antony, and Lepidus, to break off from each other and better place power in a more resolute, single direction. While one may see Octavius’s actions to dispose of Antony as being of rather underhanded political maneuvering (i.e, having the war declared on Cleopatra rather than on Antony, thereby circumnavigating the slippery slope of official civil war entirely), perhaps it was for the best. One can only imagine how different things would be in the event that Rome was taken under the clutches of Antony, assisted by Cleopatra. Personally, all I can say is that the aftereffects of Octavius’ victory led to the Pax Romana. And if anything good were to come of the Battle of Actium, then that would be the primary thing. For it was about time that Rome set down its weapons and embraced peace, if only for a time.” Titus explained, looking a little bit weary of having had to explain himself. Right now, all he wanted was a nice nap. But there was no time for such matters. There were still things that he and his cousins had to do. So, he stood straight and watched the other Recentii closely. ‘Well, Titus. You’ve already killed a man.’ he thought, furrowing his brow. ‘And it looks like you might need to man up and do it some more.... So that no one needs to resort to violence again.’
That's what practomimetic learning looks like. That's the power of the narrative.
1.12.2011
The Impact of (not) Grading
Hump day and blizzard day in New England. How about a day away from LAPIS and technology in general? Sounds like a plan!
Back at the start of the second quarter, I discussed the changes that would be happening with my Latin II classes -- specifically the different outlook towards grading. (See: The Day the Music Died) So how did the first full quarter without putting a single grade on top of a page go?
In all honesty, given the fact that I myself am wading into very uncharted waters, I don't think I could be any happier with the realistic results. My highest powered students, at first, had the most difficulty freeing themselves from expecting the "A" at the top of the page. After the first few assessments and a couple of discussions on what an "A" means (or doesn't mean), they became far more receptive to the idea that they actually didn't need that personal gratification moment to process or evaluate their own learning. Suddenly acquiring vocabulary, increasing reading skills, or even conjugating verbs became not about how this activity will impact their grade but instead about how this activity will impact my overall understanding of the Latin language.
I found that the middle-of-the-pack students began to take far more ownership and responsibility for their learning. Ensuring that their assignments were finished not because they wanted the homework check points (which ceased to exist), but because they wanted to better understand the passages in front of them. They used their class time far more effectively and, anecdotally, I'd be willing to wager that the mathematics (had I graded normally) would indicate that they improved as well.
What about the traditionally lowest performing group? Well, here's where the results are a bit mixed. A portion of this group were given a new lease on the joy of learning for learning's sake, and a new sense of pride when they themselves could see improvements -- no matter how small that they actually were. It's easy to see how hard it would be for a student (who shows improvement) but still receives a C- on a quiz or a test to ignore any gains and instead focus on the negative ("I'm still doing poorly") The other portion of this group still struggled to produce any real meaningful gains in ability or skills, despite every attempt to find more ways to bring them into the fold. But that's a topic for another day.
So, it's the end of the semester, what happened with evaluation -- after all, I do need to report a grade to the powers that be for each student. Here's how we handled this. The last couple days of the semester, each student and I had an individual conference outside of the room while the class was working on various assignments. I can honestly say that for nearly every student, I've never had a more thorough and real conversation about their own personal learning. We talked first and foremost about strengths and weaknesses, where they've seen the most growth this quarter, and what they intend to focus on at the start of the third quarter. We looked back at the goals they set at the start of this experiment and decided how and in what ways they met them. We looked at their self-evaluation 'mastery sheets' to see where gains were made and what still needs work. Only two or three students were unable to talk with much depth about their progress this quarter.
It was an amazing experience to have conversations with each student on this level. A significant number of them even went as far as saying that this was one of the best experiences that they've had in their academic careers so far. I couldn't agree more.
How did they grade themselves? Well, I had to convince a very large number of them to allow me to give them a slightly higher grade that what they thought they deserved.
Back at the start of the second quarter, I discussed the changes that would be happening with my Latin II classes -- specifically the different outlook towards grading. (See: The Day the Music Died) So how did the first full quarter without putting a single grade on top of a page go?
In all honesty, given the fact that I myself am wading into very uncharted waters, I don't think I could be any happier with the realistic results. My highest powered students, at first, had the most difficulty freeing themselves from expecting the "A" at the top of the page. After the first few assessments and a couple of discussions on what an "A" means (or doesn't mean), they became far more receptive to the idea that they actually didn't need that personal gratification moment to process or evaluate their own learning. Suddenly acquiring vocabulary, increasing reading skills, or even conjugating verbs became not about how this activity will impact their grade but instead about how this activity will impact my overall understanding of the Latin language.
I found that the middle-of-the-pack students began to take far more ownership and responsibility for their learning. Ensuring that their assignments were finished not because they wanted the homework check points (which ceased to exist), but because they wanted to better understand the passages in front of them. They used their class time far more effectively and, anecdotally, I'd be willing to wager that the mathematics (had I graded normally) would indicate that they improved as well.
What about the traditionally lowest performing group? Well, here's where the results are a bit mixed. A portion of this group were given a new lease on the joy of learning for learning's sake, and a new sense of pride when they themselves could see improvements -- no matter how small that they actually were. It's easy to see how hard it would be for a student (who shows improvement) but still receives a C- on a quiz or a test to ignore any gains and instead focus on the negative ("I'm still doing poorly") The other portion of this group still struggled to produce any real meaningful gains in ability or skills, despite every attempt to find more ways to bring them into the fold. But that's a topic for another day.
So, it's the end of the semester, what happened with evaluation -- after all, I do need to report a grade to the powers that be for each student. Here's how we handled this. The last couple days of the semester, each student and I had an individual conference outside of the room while the class was working on various assignments. I can honestly say that for nearly every student, I've never had a more thorough and real conversation about their own personal learning. We talked first and foremost about strengths and weaknesses, where they've seen the most growth this quarter, and what they intend to focus on at the start of the third quarter. We looked back at the goals they set at the start of this experiment and decided how and in what ways they met them. We looked at their self-evaluation 'mastery sheets' to see where gains were made and what still needs work. Only two or three students were unable to talk with much depth about their progress this quarter.
It was an amazing experience to have conversations with each student on this level. A significant number of them even went as far as saying that this was one of the best experiences that they've had in their academic careers so far. I couldn't agree more.
How did they grade themselves? Well, I had to convince a very large number of them to allow me to give them a slightly higher grade that what they thought they deserved.
9.23.2010
Active Assessment
 |
| The House of Caecilius |
As the teams began discussing how they would move from Sextus' house (location 51 on the map linked here) to Caecilius' house (location 34), it was abundantly clear that they had very little, if any, experience in reading maps. Among the many different emails that I received over the course of the night asking for help, one of my favorite was from a student that asked if I could just give him Sextus' address so he could plug it into a GPS-based mapping system and get the directions. In fact, more than one of the teams posted turn-by-turn directions as if they were in some sort of ancient car, complete with distances measured in meters. "Ok", I thought to myself, "this going to be a long night." Each subsequent post by the operatives indicated a further disconnect from being able to read and interpret two maps and merge data between them. In fact, most of the students threw their virtual arms up in the air and said "I give up, I have no idea what I'm looking at or how to find this house." The most creative post went to the team that said their Recentius was confused and decided to follow another one of the characters.
It was then I decided to abandon my plans for the next day and instead focus on improving our map literacy and how to use the many different online sources to figure out where we needed to go. I also figured that I would poke a little bit of fun at the fact that they were trying to use modern directions and place markers to navigate the ancient city. During the actual class on the next day we took some time to pinpoint on the Google Map where each of their Recentius characters ended up using the modern directions. One of the best "tricks" that I did was open both maps up in two separate tabs and align them up so that flipping back and forth showed how the satellite image and the charta aligned perfectly. This was going to be yet another notch in the belt of practomimetic learning; we could add geography to the growing list of skills that the whole experience improves and builds upon.
Then something amazing happened.
Two students, not content with giving up for the night, were determined to figure out where the House of Caecilius was on the charta and how their character, Octaviana, was going to get there. As the all powerful Demiurge figure looked on curiously, they traded posts in their team forum, piecing together the information from the two maps. Eventually, between the two of them, they discovered in the legend of the charta that the House of Caecilius was indeed on the map, now it was a matter of reconciling the satellite image with the other map. Around 9:50 (50 minutes after everyone else had logged off with their work for the evening completed - we have a 9:00 deadline for the online portion of the class) they finally discovered the location of the House of Caecilius and plotted a course through the streets of Pompeii to arrive at the "gignormous house on the right that would seemingly be hard to miss" (their words, not mine). You could literally see the sense of accomplishment in their words and, as the Demiurge, I made sure to offer them a hearty congratulations for their persistence.
The next day in class we traced through the logic and the thought process that these two students did the night before to figure out (together) where the House of Caecilius was and how to get there. When the rest of the class had that "ah-ha!" moment and put it all together, it was then I revealed to them that their two fellow operatives had accomplished all of this the night before, with no aid from the Demiurge.
I had no intention of spending as much time as I did on map reading skills but thankfully I was able to see that they desperately needed assistance in learning how to use that type of data to their advantage. We talked about why you couldn't always rely on a GPS device for directions and the usefulness in having the ability to read maps. Do you think that we'll be doing more travel activities during the rest of the practomime during the year? You bet.
This wouldn't have happened in a traditional setting. We look at maps all the time, discuss certain features, landmarks, points of interest. However, never before have I required teams of students to get from one point to another as a part of an ongoing narrative. Without the narrative, without the practomime, I would not have had the chance to actively assess the students as they floundered about getting from the House of Sextus to the House of Caecilius on that evening. I wouldn't have entirely adjusted my lesson plan to compensate for a deficiency in a skill that goes beyond the Latin classroom. I wouldn't have even discovered just how little experience they had in reading maps.
And, for those two students, they wouldn't have felt the pride that goes along with accomplishing a challenging task collaboratively as a team.
Subscribe to:
Posts (Atom)