Thoughts on the ALIRA Exam

I just wanted to share my comments about the ALIRA Exam; we gave the exam to our Latin 1 and Latin 3 students today.

The exam is very short; the website says it has a 50 minute timer. Most of the Latin 1s were done in under 10 minutes, most of the Latin 3s were done in under 15. I'm a little concerned that there aren't enough tasks before it ends the exam in order to get a true sense of where the student is at. Most of my I2 and I3 scored students saw 12-14 questions, my I4 and 5s saw about 18. Those in the novice range saw fewer than 10.

The Latin sources used were indeed varied in context and in time. I do question the design choice to provide this weird papyrus background with a fake script font for the bulk of the passages. My students said it was distracting and I'd definitely agree. There's really no reason to create a faux-situated experience on an examination in that manner. You can see some examples of it here in the ALIRA sample test.

There was a big show-stopping bug which impacted a handful of students (I did email Language Testing International about it, hopefully I'll hear back soon) and invalidated their scores. I had a couple of solid Latin 3 students score N1 (and one even score BR - below the range). Interestingly enough, all four of them ended on the exact same question (about 13 questions in) and no one else saw that question in their rotation. I hope that the question can be fixed or removed as others take the exam.

In the end, many of my students walked away with a bolstered confidence level and affirmation that they were on the right track. My Latin 3s were, by in large, right where I thought they'd be knowing the students. Most were I2 and I3, a few I4 and 5.

The Latin 1s were the surprise of the bunch; except for 3 N1s (and I suspect a test error here, too), everyone was I1 or above with a large cluster at I3. In essence, ALIRA is telling me that our Latin 1 students are at the same place our Latin 3 students are in terms of ability to comprehend a text. I need some time to digest this data and think about how a few things may be impacting those scores; but my Latin 3s are really the last vestiges of my former life of reliance on a more grammar-translation-reading approach and the Latin 1s, in addition to having a lot more CI-type activities embedded in their daily instruction, also have the added benefit of 2 full years of refinement to Operation LAPIS aiding them as well.

Overall, though, I'm very pleased with the ALIRA exam, the time to administer it, and the information (if it is indeed accurate) that it provides. I'm not convinced it's a $10 test, however. That price needs to come down so that I (and others) can administer it program-wide each year.

In an effort for those of us using ALIRA to gauge where our students are at as a whole, I want to start compiling data for exams given this year. I created this Google Form for anyone who wants to contribute:

https://docs.google.com/forms/d/1sx87t04SkDYByBWXKghbmXJHm5iAq3Xjc7Nuf_uvSVo/viewform

Names and schools won't be published (included that space, though, to help ensure that we're not getting false data), only raw numbers for proficiency at each course level.

If you have had experience with ALIRA, it would be great to share your thoughts, so please do!