Do you need a rubric?

Someone has recalled a book I borrowed from the University Library, so that has made me look at the book and consider why I borrowed it in the first place. The book is Introduction to Rubrics by Stevens and Levi (2013). I think I borrowed this book because I knew that my colleagues who are interested in curriculum desig,n and in assessment, had been asking me about rubrics. So before I return the book to the library I’ll share just a little bit with you.

Stevens and Levi pose the following question, ‘How to you know if you need a rubric?’ Then do on to say ‘One sure sign in if you check off more than about three items from the following list (the list contains 13 items – so I’ll just share small sample with you):

* You work with colleagues and collaborate on designing the same assignments for program courses, yet you wonder if your grading scales are different.
* You’ve sometimes been disappointed by whole assignments because all or most of your class turned out to be unaware of academic expectations so basic that you neglected to mention them (e.g., the need for citations or page numbers).
* You have worked very hard to explain the complex end-of-term paper, yet students are starting to regard you as an enemy out to trick them with incomprehensible assignments.
* You are starting to wonder if they’re right.

Clearly Stevens and Levi have a sense of humour. It looks as though I’ll have to re-borrow this book at some point in the future because I have not made through to the final chapter on ‘Rubrics and Program Assessment’ and yet, I am liking that chapter already:

Want to know how to clear an entire building of faculty offices in less than two minutes? No, it has nothing to do with smashing glass, pulling a handle, or yelling, “Fire!”, Simply murmer the words “progarm assessment,” and watch as the halls and offices empty. For many faculty members, those two words bring up stress-filled memories of hours of paperwork; even more hours of pointless, conflict-ridden meetings; and at the end of it all, a report that bears no resemblance to they do, what they want to do daily, and what real difference it will make in student learning. Program assessment doesn’t have to be like that. It isn’t always like that, but we all remember cases when it has been.

I’ll keep you posted on what I find out next!


Leave a comment

Filed under Uncategorized

Do you want all the flavour to be revealed in the first lick

I spent some time talking to colleagues about writing their program/course and unit/subject student learning outcomes. We’ve become a bit distracted with this because writing good Course Level Learning Outcomes is something we all have to do as one of the steps to ensure that the qualifications offered by our university meet the Australian Qualifications Framework (AQF) requirements. Some of my colleagues indicated that they felt a tension between needing to specify student learning outcomes in advance and leaving open the opportunity for student learning that fell outside that indicated by the intended learning outcomes. So this set me to thinking (or rambling) . . . 

Is you subject like a journey? Are you the tour guide? Do you want no surprises for your students? What about your teaching team? What about your professional colleagues who manage timetabling, teaching spaces (virtual and 3D), examinations and other logistic matters?

Perhaps your students are taking a journey with you? You know the terrain, the language, the cultural practices and the highlights (not to be missed) and the low lights (go there only with a good guide, plenty of light and a translation dictionary).

Perhaps your teaching team and professional colleagues are the travel agent, flight crew and travel guides and your students are the travellers. They need, and expect, a safe and informative journey. Perhaps the journey will be transformative for some students (or even for some staff).

Maybe some students do not want to take the journey – they want to see if they can experience the flavour (or even become a connoisseur) with just one lick. But they will only get the fast fading experience of eau de parfum. They might ‘think’ that they have the flavour – but they will have no nosomic memory of it, they won’t be able to recall it and even if it is returned to them it will evoke no visceral or cognitive response – it has not become part of their memory.

So, what of those students who do embark on the journey? What should be in the tour brochure and itinerary to inform the intending student and to ensure that this is an appropriate journey for them at this point in time and that they are well-prepared for this journey (so that they remain safe and so that they can receive the full benefits available from this journey)?

Are you concerned that if you reveal details of the likely, the hoped for, benefits that this will detract from the experience? Are you concerned that in trying to ‘get’ the promised benefits (the intended learning outcomes) students will miss some unintended outcomes (the serendipitous experiences)? Are you concerned that in having to document, in advance, the hoped-for benefits, that students will blame you if they do not achieve them/obtain them?

Do you think that you’ll be tempted to include only those benefits (intended learning outcomes) that you can guarantee for all students (and therefore write learning outcomes that have a low cognitive demand, rather than those that have required higher cognitive demand)?

Is there another way to look at this? If, instead of seeing students as customers (travellers for whom you prepare a low-risk itinerary with guaranteed outcomes over a pre-specified time frame) should we instead see students as fellow travellers, ones who come with their own wealth of experience and personal motivations? Could we consider that we have as much to learn from students as they have from us? So while we are the ‘facilitator’ of the travel plans (because that is our job, we have been there before and we can broker deals with the professional support staff and the in-country personnel we need to deal with, and we can help create synergies and economies of scale that group travel can provide) we do not carry the sole responsibility for reaching the planned for destination. It is a shared responsibility and, as with any Kontiki tour, part of the adventure is finding out about one another and re-negotiating what it is that we want/need from this particular experience and what it is we are prepared to, and able to contribute?

So, my advice would be to write your Subject Outline (and particularly your Learning Outcomes) with this philosophy in mind and perhaps it will be less frightening and more fun. As connoisseurs know, there might be some taste in the first lick, but it is not the full flavour and it is not the one that endures, creates a desire for more, or improves with time and in combination with other flavours. My subjects will be those that develop the palate and take your ‘flavour brain’ to places it has never been before, but you will often want to re-visit.

Leave a comment

Filed under Uncategorized

Wicked competencies

After talking to Beverley Oliver yesterday about her ALTC National Teaching Fellowship (see earlier posts) I decided it was about time that I followed the advice of my colleague Gordon Joughin (now at the University of Queensland) and find out about ‘Wicked Competencies’.

So, I’ve found a paper that looks interesting and recommend that others read it too.

Peter Knight and Anna Page in their Report to the Open University’s Practice-based Professional Learning Centre The assessment of ‘wicked’ competences say

‘Wicked’ competences are achievements that cannot be neatly pre-specified, take time to develop and resist measurement based approaches to assessment. On the basis of knowledge of assessment practices in higher education in general, it was anticipated that there would be acute problems assessing this class of outcomes. And they are also important outcomes of higher education, since they are widely valued by employers and smooth the path of study and other forms of research.

If you are as intrigued as I was you can read more:




Filed under Uncategorized

Professor Beverley Oliver new ALTC Fellow

There is exciting news with the announcement that Professor Beverley Oliver from Curtin University is a 2011 ALTC National Teaching Fellow. Several University of Wollongong staff will recall Professor Oliver’s visit to Wollongong in November 20101. The University of Wollongong is one of eleven partner institutions on this project of Beverley’s. Several academic units at Wollongong may be interested in this project. A short extract of information on Beverley’s Fellowship is pasted below and a link for further information provided at the end of this extract.

Conversations about graduate capabilities inevitably turn to standards: academic staff, business and industry, the community, students and graduates seek clarity on the level of achievement required for safe practice and professional readiness. Course (program) leaders, students and industry partners are often guided by predetermined lists of generic attributes, professional competencies and outcomes. However, many seek clarity about the level of performance required during the course, at graduation and beyond (for example, how well a journalist or pharmacist is expected to be able to communicate at graduation). In addition, in an increasingly evidence-based culture, the sector is seeking new ways to assure the achievement of such standards.

This fellowship proposes to engage curriculum leaders of undergraduate courses from any discipline to work with their colleagues, industry partners, students and graduates to:

  • define course-wide levels of achievement in key capabilities, articulated through standards rubrics
  • implement strategies to evidence student achievement of those standards (through student portfolios and course review processes, for example)
  • share the validity, challenges and opportunities of such approaches through scholarly publications.

Colleagues are encouraged to access an introduction to these concepts and join a community of practice and scholarship at <>.

Leave a comment

Filed under Uncategorized

ALTC announce four new projects on assessing graduate outcomes

The latest edition of the ALTC (Australian Learning and Teaching Council) News  (18 April 2011) announces four new projects on assessing graduate outcomes. Here’s a short extract and a link so you can read more. They all look interesting.

‘Four new ALTC-funded projects will focus on quality assurance and assessment of graduate outcomes, with almost $1 million dollars to be invested over the next two years.

Each project covers different territory in terms of research methodology and focus but all ultimately aim to better connect assessment to the needs of industry and students as well as to the broader goals of universities themselves.’

Leave a comment

Filed under Uncategorized

Why should we consider mapping ‘discipline standards’ in the curriculum?

One of the topics I get asked about when I provided advice to faculties and individual academics about curriculum development and curriculum mapping is the place of the Discipline Standards (a project of the Australian Learning and Teaching Council ALTC) in curriculum. At least one discipline at UOW is going to map their curriculum to their discipline’s Threshold Learning Outcomes and they plan to use the Curtin University Curriculum Mapping Tool to do that.

For one view on how the discipline Threshold Learning Outcomes (TLOs) might influence curriculum review and design you might like to watch a series of four podcasts (each one is between 3 and 6 minutes long) developed when Professor Lynne Hunt (Pro Vice-Chancellor, University of Southern Queensland) interviewed Dr Carol Nicoll (CEO, Australian Learning and Teaching Council). The topics covered are:

What do teaching standards mean?

Do standards lead to standardisation?

What is the relationship between teaching standards and professional accreditation?

What should universities do to accommodate teaching standards?

Leave a comment

Filed under Uncategorized

‘A map is . . . an arbitrary selection of information’

I was reading the column Armchair Journey in the Travel Section of yesterday’s Sydney Morning Herald when my attention was captured by the opening sentence, ‘It seems such and obvious observation: “A map is in its essence and intent an arbitrary selection of information” ‘. The reviewer of the book Infinite City (by Rebecca Solnit, University of California Press), Bruce Elder. It’s not a book about curriculum mapping (surprise!), but it still made me think. Elder describes what Solnit does in this book as the creation of a ‘personal atlas’ of the ‘principal landmarks  and treasures of the region’ of San Francisco, California. Each map is constructed based on a construct that Solnit herself has identified: butterfly species, libraries, coffee, and so on.  As the reviewer, Elder, points out, ‘The possibilities are endless, Infinite City offers a new way of thinking about any location’. It sounds like a great book.

Most of the discussions I’ve had with people about undertaking curriculum mapping have focussed on mapping their curriculum that allows them to demonstrate key features and relationships that must be identified as part of a curriculum review. However, it is probably worthwhile to think about other signficant features that the should be identified. In this way curriculum mapping could be a useful tool (and not just a compliance requirement) to guide curriculum review and development.

Leave a comment

Filed under Uncategorized