Making the Grade in CBE

I recently reviewed Gregory Arbuckle's Educause article on his challenges in implementing Competency-Based Education (CBE). I was pleased to hear his passion for CBE, and his comments about how CBE can help to drive alignment between industry needs and educational outcomes - comments that those struggling to act on Ray Ivany's ONE Nova Scotia report (Now or Never) would be well-advised to heed. Arbuckle also spoke of the frustration of trying to manage grading basis in a CBE environment. 

It is great to see the resurgence of interest in Competency Based Education, but if there is a return to this very effective model and pedagogy, then I only wish that those adopting it would also understand that there is little opportunity to truly resolve the dichotomy of grading practice between CBE and academic approaches. They simply don't typically talk about the same notion of performance. That being said, there certainly are ways to address the needs of academic institutions to interpret learner evaluation in a currency that is not native to CBE. 

From my understanding and experience in CBE, competency performance in CBE can be approached using a mastery scale that is associated with the DACUM model (of some time back) - a scale that assigns a value from 1-4 on demonstration of competency (see Mark Rosenburg's blog piece for a relatively recent discussion on this scale). 

In our college (similar to Arbuckle's situation) we also struggle with the need to communicate with academic institutions and individuals who think that 76% has a particular meaning regarding what a learner can do. The College requires that all learners must demonstrate minimum (entry-level or novice) competency in all outcomes of the program and course (as defined by industry). Since we sometimes have to report grades, then we also assume that such competency would meet a minimum grade of 60% or 70% (the program minimum pass mark). Students who are able to demonstrate a higher level along that scale or who come to us with experience or even mastery of an outcome/competency may achieve grades of 80 or even 90. However, this is just a way to appease those who don't get the idea that a 1 or 2 is a perfectly acceptable level of competency for entry level employees. Since 100-based grading is irrelevant in a true competency-based pedagogy, matching up the breakpoints is really - well, academic.

An interesting new approach may be found in the use of portfolio and badges (Mozilla's Open Badges initiative) and I think this holds great significance for CBE institutions. Having authentic, evidence-based, experiential assessment of competency should be welcomed warmly by industry - except, perhaps, by those populated by traditional academics.

Graduate Profile

I am sometimes intrigued at the latitude taken by some institutions and individuals of influence within some institutions around the interpretations of certain academic nomenclature for which a wide degree of interpretation is not intended. At the same time, I recognize that individual institutions must use their own values and vision as a lens through which they view and interpret various philosophies, methodologies and pedagogies and that there is some room for interpretation in much academic terminology.

In my current organization, there seems to be reluctance to adopt a consistent meaning around some very important academic notions, although we are rich in academic philosophy, values, and vision. Given that richness, it seems we should be able to use them in resolving common nomenclature. The most recent sticking point seems to be the concept of a "graduate profile".

So, I went out to seek some definitions of graduate profile and program portfolio using a variety of sources including literary searches, academic policy review, and a couple of internet search engines. In that research I came across many definitions of graduate profile as used by other institutions. Generally they fell into one of the following three general approaches:

 

1. An individual graduate student’s profile – a presentation of their learning and competencies, unique to that student. (we may call this a student’s portfolio)
2. A description of the intended outcomes for a graduate of a particular program including both the specific technical competencies and the workplace skills (literacy, numeracy, collaboration, etc.). This approach where observed sometimes pointed to likely graduate employment prospects. (this may be referred to as the program curriculum or prospectus)
3. A vision of the graduate of a particular institution – encompassing the general skills of all graduates of the institution regardless of the specific technical skills (also sometimes referred to as college-wide outcomes)

I was particularly taken with a prologue to graduate profile taken by Rift Valley Academic in central Kenya:

 

At the beginning there were two possible approaches: to develop a document of graduate outcomes that all students should achieve; or to develop a document of the ideal graduate.  The committee chose the second approach.  […]

The ‘Graduate Profile' is:

• An ideal toward which we direct the student.
• A tool to assess the extent to which we are accomplishing our vision.

The ‘Graduate Profile' is not:

• A graduation requirement.
• A cookie cutter.
• A  tool to pass judgment on individual students.
• A student self-assessment standard.

It seems to me that as with most process and systems advancement, the first question to answer is “What do we want this application, process, or system to do?” Once a clear purpose is established the other requirement pieces – regardless of the terminology used – will fall into place. Perhaps we just need to define what we want graduate profile to accomplish at our institution?

Massive Open Online Course

Just signed into #PLENK2010, (connect.downes.ca) the newest MOOC offering from George Siemens, Stephen Downes, et al. The course is just getting underway but already an interesting group of folks introducing themselves in the intro forum. Lots of assorted interests and points of view; still hoping to catch some folks who may have a bent towards eportfolio and the aggregation of personal learning environments/networks into a "central portal" for assessment, career planning, showcase, etc. in such a way as to offer value, integrity and opportunities to assess learning (and teaching) performance.

I suspect there will be some great opportunities to talk about some of the same issues that are plaguing the portfolio/eportfolio world in areas like identity, privacy, integrity, honesty, etc. Good stuff! More to post as we proceed through this innovative 10-week open course.

Re-connected

Rather than making excuses and apologies for a ten-month absence, I think I'll dive right in and get back to this. Much has been happening with me over this span...but it has circled back to this blog as my opportunity to reflect, read, write, reflect some more, etc. Just before my recent vacation, I attended (http://www.nscc.ca/) NSCC's Introduction to Portfolio Learning course, and in that experience re-affirmed the importance of deep reflective thinking and writing as a critical learning cycle. I'll have more to say about that course and its experiences in future posts.

I have signed up for PLENK 2010 (Siemens, Downes, Cormier, et al) and looking forward to the mid-September start of this open course on Personal Learning environments and Network Knowledge. I shared a seat and some discussion with Stephen Downes on the way to St. John's on August 4 and later caught up with him doing his photo walkabout downtown St. John's. It reminded me that I have a great deal of material to be edited and posted on Class in the Cloud podcast - from Downes, Siemens, Trent Batson and others.

EDUCAUSE, TEKRI and others are currently bringing some focus to Learning Analytics - a subject near and dear to me for the past couple of years as a result of my work on program renewal and matrices for program assessment (of feasibility as well as quality). EDUCAUSE has opened SEI (Seeking Evidence of Impact) while TEKRI has just announced a conference on learning analytics. I'll be looking forward to engaging in both of these and interested in any other like-minded conferences, discussions, presentations.

A Dollar...and change

I am thinking today about the nature of change in institutions like a University or Community College and the challenges that "change champions", and change agents face in organizations that seem sometimes hard-wired to resist change at all costs. So much of my work and my life seems to be about facilitating change and working through the resistance that change might present. A few chestnuts to start with:

Q: How many academics does it take to change a lightbulb?
A: CH-CH-CH-CHANGE? What do you mean change!!?

"Why is it that faced with the option of either making change or proving why it isn't needed, so many academics will get busy with the proof?"

Sorry I can't provide reference for either of these - they aren't mine, but I don't have the originating source at hand. However, I just finished reading a novel (State of Fear) by Michael Chrichton presenting an alternate view of the current take on climate change. In the epilog at the end of the book, Chrichton pontificates on a lot of issues relating to the current state of science research and politics. Some of it is pretty controversial, but I particularly liked one comment relating to the inability to change the model of science research. "The world changes; idealogues and zealots don't". He also states "I am certain there is too much certainty in this world."

It has been a point of interest for me for some time to explore why some decision-makers in academic areas are resistant to change (see related articles on my ePortfolio research site or my Class in the Cloud podcast with Stephen Downes). In my discussion with Steven Downes, he cautions that I should not use terms like "technophobia" or "fear of innovation", suggesting that such folks have no reason to fear change - rather that there are valid (and often economic) reasons for resistance in many cases. Agreed that there are great reasons not to simply change for change itself, but it seems to me there is also plenty of inertia when it comes to change that would be very obviously beneficial and that is where the notion of responsible technology conservatism does not help explain the reluctance to embrace change.

King Whitney Junior presents an interesting take on change and the emotions that accompany it: Change has a considerable psychological impact on the human mind. To the fearful it is threatening because it means that things may get worse. To the hopeful it is encouraging because things may get better. To the confident it is inspiring because the challenge exists to make things better.

I'm the confident kind, I guess.

Halifax Podcamp 2009

7:00am is an early start for a Sunday morning at Halifax Podcamp 2009, but I think that it was time well spent. For a free event, I feel I got a lot of value. I started out "camping" with Hal Richman on Convergence of Business and Social Media. The message? Unsure, but it seems that some folks have had some success utilizing social media tools to accomplish business objectives, but many find it difficult to get past the hype and make a real connection to do more than a marketing exercise (that might be done better in other formats). Said Hi to Steven Downes who seemed to make pretty well the same rounds as me for the session choices.

Session II: MediaBadger - The State of Social Media in Atlantic Canada. As the barriers to entry to publication (editors, publishing houses, costs to publish) gets ripped down by social media tools, marketers are finding new ways to get to the 18-34 demographic that heretofore has been difficult to reach - but will become highly political and important over the next decade. Meantime a gap is clear between internet users in the lowest wage groups (47% actively using internet) and the highest wage groups (91% usage). Interesting stat - 67% of Canadians watch TV and are online at the same time. I thought it was just me??

Session III: I decided to go with Chris Campbell's "Small, Specific, Real: Storytelling" presentation. I note that he lost a bunch of folks right off the draw when he exposed his technology for the presentation - sticky notes and the audience. Too bad for them - although Chris' style may be a little disjointed for some, he had some great notions about how to tell a story in a medium like blogs or tweets. Keep the storyline tight (small), the delight is in the details (specific), and genuine things that mean something to you (real) will always make for compelling stories. A couple of other good notes in the margins - like understanding that you don't have to tell people the meaning of every story and that entertainment is still a social construct - good stuff. I will try to find his podcast (Bad Metaphor) and give it a listen.

After lunch I took in the keynote with Andrew Baron (RocketBoom) - a bit of a waste of an hour and a half, but a couple of good (albeit altruistic) takeaways: If you find yourself explaining the notion of Web 2.0, you are probably talking with the wrong audience. Also - and this really resonates with me - aggregation is the true promise of the semantic web. His final notes were no real news: To be a success in social media marketplace you will need to capture two of the following characteristics: be the first; be the best; be the most unique. Good thing this wasn't the only draw for the event.

Last session of the day was a chat with Craig Moore (Spidervideo.tv) and was chock full of great production and post production notes for adding video to web presence. Having a long drive out to the countryside I didn't stay aaround for the final framing - hoping it was taped and will be available/ linked on the podcamp wiki.

ePortfolio - Mash-up or Proprietary System?

Whistle Stops: Those who have been following my research know that I have deliberately focused on ePortfolio systems, eschewing any consideration of fully decentralized approaches such as Helen Barrett's Google mash-up and similar solutions having no central portal for cohesion.

Barrett has been a champion of the "learner as central character" in all aspects of portfolio learning and in that respect and in the great work she has done to help focus institutions on strategic approaches (that start with understanding the purpose of portfolio before implementing technology) she'll find only admiration from me. Barrett's ideas are driven by a dedication to digital story-telling and to the notions of writers like Thomas Friedman (The World is Flat) - "small pieces loosely joined". Again, I'm a fan of both of those directions. My opposition to the decentralized approach parallels the fundamental argument between applied research and theoretic research - and perhaps between the community college and the university - that puts such idealistic approaches in isolation of the reality of learner motivation and institutional accountability. In other words, in theory mash-ups sound like the perfect portfolio technology. In practice, they may simply de-rail the opportunity for ePortfolio to plug into the semantic web to bring critical value to the learner, institutions and society generally.

Academic ePortfolio research is often (and necessarily) conducted hypothetically. Ms. Barrett has certainly been more "hands-on" than many - having populated dozens of Portfolio tools to present experiential evidence of their efficacy. It is in the fact that most learners have no interest in populating numerous portfolios, that my first argument is grounded: collecting and populating ePortfolios is hard work if done comprehensively. ePortfolio systems - backed by industry standards - promise the opportunity to transport robust, complex portfolio data from application to application without having to ask learners to re-submit and validate entire repositories of evidence of - and reflection on - learning.

Secondly, academics do not usually concern themselves with issues like the total cost of ePortfolio implementation and the level of accountability Senates and Boards of Governors require to rationalize such investment. Open, uncoordinated systems leave no aggregate data to track accountability and therefore may be unattractive, if not unsustainable, to governance bodies. In the long-term, open decentralized approaches may be able to stitch together disparate data to create portable portfolios but in the short run, the successful business model to grow ePortfolio will require that standardized, aggregate systems return consistently comparable results from institutional investment in ePortfolio.

So how does one support digital storytelling theory and decentralized web 2.0 applications while arguing for institutionally hosted, centralized ePortfolios? As a technologist, I don't assume that they are mutually exclusive. Barrett's ePortfolios 1.o vs 2.0 argument in 2006 presents an apparent dichotomy of the monolithic ePortfolio (1.0) and the decentralized one (2.0). However, it suggests that parameters like competencies and standards can't accommodate learner creativity; that "authority figures" are outside the community of learners; that blogs and wikis are unstructured (unlike forms); that accountability somehow is not about learning; and that proprietary systems cannot leverage open architectures. Trent Batson's recent Campus Technology article outlines a more pragmatic notion: to maintain artifacts wherever learners choose (MySpace, YouTube, Google Apps, Facebook, etc.) while the centralized ePortfolio portal provides cohesive tracking for assessment and evaluation.

Reflections: The digital divide for eportfolio will not be between the technology have and have-nots but may be found in strategic gaps in solutions designed rather than inspired by idealists. If we are to follow the best practices of eBusiness consulting in defining ePortfolio, we should let the academics provide the vision and functional specification and then get them out of the way as technologists work to make it so.