Loosely coupled assessment

Here’s the thing it’s 2000 all over again. Eportfolio is the new LMS.

Watching a recent vendor presentation I thought “I can’t believe this is happening again.”

That single phrase. In a loop. In my head.

Because remember — this happened once before. The LMS vendors came in with an assessment and management tool, and told us it was an elearning solution. At the time, I was on the other side of the equation, with a company trying to sell award-winning goal-based scenario software to colleges who were saying but we already HAVE an elearning solution. It’s called Blackboard. Or WebCT. Or whatever.

And so Blackboard, an assessment and management tool, determined the pedagogy of colleges for eight or so years. Because teachers wanted to import rosters, we put students in a closed box and told them it was elearning.

When it wasn’t. The truth is the kids were doing more elearning on MySpace than in Blackboard.

How do we avoid it again? How do we avoid imposing something that is just pedagogically WRONG on a new set of students because we need to meet some institutional assessment needs?

There’s only one way — loosely coupled assessment.

If we are going to talk assessment, we are going to have to segregate it. Your assessment tool should ONLY assess.

We don’t need to talk more about student needs wth vendors that supply assessment tools. We need to talk to them less about student needs. It’s not their business.

Literally: it is not their business.

In fact, we should remove student needs entirely from the equation.

The students know they can get far bettter solutions to their problems for free elsewhere. They don’t need a eportfolio system to post their resumes on.

So enough of letting assessment vendors tell us what facilities we will be forced to use in their walled garden, and expecting us to be excited about it. Enough with assessment vendors selling us “environments”. What we should be doing is describing the the enviroment that might exist — students using WordPress, Blogger, S3, GDrive, email, messaging, etc. And then we should ask if they have a tool that can evaluate that. How will their tool interface with the learning environment we’ve constructed?

Anything else is insanity.

EPortaro demonstration today (any help?)

I’m attending an EPortaro demonstration in about 50 minutes.

If you read this blog at all, you can probably guess what I think about such eportfolio solutions. It’s 2000 all over again, with vendors coming in to save us from the big, bad internet.

Still, my opinion is probably the minority one on my commitee, so there’s a good chance we are going to pick SOME eportfolio vendor, despite my efforts.

If you know anything good or bad about EPortaro in particular, please email me at mcaulfield at keene edu. Or leave a comment.


Where will the wave come from?

I love talking the theory, but it’s even nicer to see practical notes from people implementing solutions. From a recent post over here, some WordPress MU as class-space experimentation

Teachers are finding WordPress MU easy to use and I’m very happy to see that. Currently, Teacher Assistants are recording students as they read their writings in class using Audacity. We are using inexpensive mics with noise canceling, and I have to say, I’m impressed with how well they work. It’s not easy to cut out the ambient noise in a working first grade classroom.

That’s right. A first grade classroom.

I’ve been a frequent critic of primary and secondary education, and that’s unlikely to stop. But I’ve been impressed in the past year with how much faster things seem to be moving down there than up at the university level.

It’s not just scattered notes like the one above. The percentage of thought leaders in the Learning 2.0 space that are focussed on K-12 is extraordinary.

Why? One would think if you can run a blog and wiki with first graders that surely this should be cake for a university classroom.

More as a way to start this conversation, here are a few hypotheses:

1. K-12 (and particularly K-6) does not have the subject problem — there is no issue that writing belongs in one discipline, video in another, and history or math is seperate from each. Holistic approaches aren’t thwarted by an org-chart that divvies up the student.

2. K-12 is behind on the LMS wave, and having not been infiltrated by LMS vendors, they are more able to think out of the box, rather than in terms of what new LMS modules are available.

3. There’s just more teachers than university professors, which creates the critical mass needed to get a movement going.

4. They don’t have a developed IT department or large IT budget — and hence are able to experiment more with an ad-hoc bricolage of tools, especially free ones: i.e. technology decisions are not treated as budget decisions.

Those ideas are all possibly wrong — but I’d love to hear other takes on this phenemenon. Unless higher education gets its act together, it is quite likely the college freshmen of tomorrow will be entering a far LESS enlightened tech environment than the one at the high school from which they came.

Electronic Textbooks and CommentPress

Via bavatuesdays, I learn of CommentPress.

Obviously there are other non-WP group annotation tools. What’s really striking to me here, however, is how powerful the fit is between the CommentPress approach to text and the best bits of traditional literary exegesis.

So great is the fit, as a matter of fact, that I half wonder if CommentPress could become the first step toward faculty blogging — rather than the other way around…

Goal-based scenario/simulation vs. learning 2.0

The most invigorating job I ever had was working for CognitiveArts programming learning “simulations”. Founded by Roger Schank, CogArts was truly a company with a mission — to revolutionize education through technology rather than simply extend the current system. And we pushed the envelope in every way we could. I worked with a large team of programmers whose goal was to make the ultimate Choose-your-own-adventure multimedia learning experiences.

The core idea was simple: people learn by doing, so learning should simulate doing in a low risk environment. Schank’s favorite talking point was this “Which would you rather your airplane pilot have — 90 hours of the flight simulator, or 90 hours of book study?”

Simulations would generally lead a person through a “goal-based scenario”: perhaps as a Governor’s economic advisor they had to make decisions for a hurricane torn state on things like price controls and rationing, and observe the effects of the action. Perhaps they had to negotiate a house price as part of Harvard Business School Publishing’s Negotiation class.

The key to the system was failure-based learning paired with just in time instruction. Students would be encouraged to develop expectations about what would happen as a result of their actions. When they failed, they would be provided with context-sensitive instruction, and encouraged to try again. It had been shown in a number of studies  that by providing the bulk of the instruction after failure that you could get retention of information significantly higher.

The system was later copied (often poorly) by other corporate training companies, and is now a pretty standard offering of most custom elearning vendors (although I would argue that the desire of many vendors to push such modules into a one-size-fits-all assessment harness profoundly degraded the experience — at CogArts we built an LMS that was precisely tailored to the needs of our scenarios).

This autodidactic gaming approach to elearning seems miles away from the PLE and the Inverted LMS (I still haven’t quite resolved if those are the same thing yet — please excuse my transitional use of both terms). The Inverted LMS is inherently social and collaborative; the CogArts model was solitary and self-taught. Indeed, if there was one flaw with what we did at Cognitive Arts, it was probably that in the move from CD-based non-networked learning to web-based instruction we were not radical enough in our rethinking of the social element of education.

Despite that, I’d argue that simulations are very close to the PLE/Inverted LMS in theory. Why?

Because both focus on learning by doing. Where there is high-risk to real life failure simulations make a lot of sense. And where the definition of success in a field or task is very narrowly defined, simulations shine. The flight simulator, one of the first computer applications ever built, still remains the model here.

But the web has introduced us to plenty of low-risk ways to engage in disciplines. And that’s where the new approach comes in.

An example? At CogArts, one of the apps I admired most was the “Is it a Rembrandt?” simulation, which provided students with detailed pictures that could be faked paintings or undiscovered Rembrandts. The students, through learning about Rembrandt’s style, had to make the call. Experts were there to give them the just in time instruction should they fail — explaining this or that about brush strokes or subject matter.

I’d still pay good money to use that sim — I think it remains a wonderful way to learn, and one that appeals to our gaming culture. Put software like that in a current high school, and you’re going to blow the doors of education. In a good way.

But what is striking nowadays with the web is how it supplies plenty of real low-risk problems for students to engage in. The Rembrandt simulation was built during a mid-90s rash of discoveries that certain Rembrandts were fakes. Ten years later if such a thing happened, there’d be a good chance you could get hi-res photos of detail from the fakes, if you asked nicely.

So what happens then? You gather your students, you put up a wiki and series of student blogs, you roll your sleeves up, and you get your class analyzing the paintings. Google becomes your just-in-time learning application, which is cool, because that’s what your JIT solution will end up being in real life. Success or failure is determined, as in life, somewhat fuzzily by the reaction of the experts in real life: if you can get them to engage with your work at all, that’s a high level of success; if they actually start agreeing with you or noting things as valuable insight, even better.

I miss both producing and playing with the Schank software, just because of how much fun it was, and if I could buy those titles shrink-wrapped from the local Staples today, I’d spend my own money to buy a title a week. Heck, I may go home tonight and play the Cable & Wireless simulation, which I still have a disc of somewhere. In a perfect world the government would fund more of these sorts of simulations.

But the brilliance of the internet is how much it matches, for a certain subset of problem, the perfect learning environment CogArts was simulating in its courseware. As with the simulations, on the internet you can try out ideas without much risk, you can get information from Google on a Just-in-Time basis, and you can talk to experts about the validity of your decisions. And, yes, it’s a lot fuzzier, and I certainly don’t want my pilot to have put in 90 hours of BLOGGING, but for certain types of learning (and possible for most learning), it’s a preferred method of engagement.

You’re already out there

This just in:

July 25, 2007 (Computerworld) — Millions of documents, both government and private, containing sensitive and sometimes classified information are floating about freely on file sharing networks after being inadvertently exposed by individuals downloading P2P software on systems that held the data, members of a House committee were told yesterday.

Among the documents exposed: The Pentagon’s entire secret backbone network infrastructure diagram, complete with IP addresses and password change scripts; contractor data on radio frequency manipulation to beat Improvised Explosive Devices (IED) in Iraq; physical terrorism threat assessments for three major U.S cities; information on five separate Department of Defense information security system audits.

I’ve been wondering for a while in whether Google Docs would be a good choice for the college’s Content Management System. It’s a wonderfully simple way to share docs, it’s cross-platform, remotely accessible, and free, as in beer.

One of the objections I’ve heard while kicking this idea around is that it might be dangerous to have our documents “out there”, in the Wild, Wild Web.

I’d actually argue the reverse — anything on a employee’s machine is already out there: in our P2P world, the web is us.

Google Docs (or something similar) actually has the advantage that it dissuades users from downloading materials to their hard drives, the major source of leaked materials. Combined with the right cache settings and some SSL action, it could be, I think, a far more secure environment than what we have now.

The one catch? We need the SSL; Google Docs seems not to have it. But perhaps it’s coming?

In Which I Meet Our (Other) Allies

So, I’ve just stumbled into a gold mine. Via an inbound link from Stephen Downes, I’ve discovered that much of what I’ve been calling an inverted LMS has been called elsewhere a PLE (personal learning environment):

Helen Barrett receives an email from Mike Caulfield describing an Inverted LMS, which turns out to be the PLE, independently discovered. More here. She also gets a note from a graduate student, who writes, “I’m trending towards the view that the system we will end up with will use RSS to expose content, tags to organize it, and open ID to selectively share content with certain people.” Yes, as people look at the potential of online technology, they begin reaching similar conclusions. Independently, autonomously.

And it’s true! There is much overlap. But just as I’m about to object that the Inverted LMS goes further than the PLE, I find this post via connections to Downes: Leigh Blackall’s Die LMS Die! You Too PLE! And stuff like this warms the cockles of my heart. All the cockles. Every single one:

Question to the PLE: Why do we need a PLE when we already have the Internet? The Internet is my PLE, ePortfolio, VLE what ever. Thanks to blogger, bloglines, flickr, delicious, wikispaces, ourmedia, creative commons, and what ever comes next in this new Internet age, I have a strong online ID and very extensive and personalised learning environment. Actually I think the PLE idea is better envisioned by the futurist concept known as the Evolving Personalised Information Construct (EPIC). I think we already have EPIC, so why do we need the PLE?

OK — apart from the fact that his was written over a year and a half before, and that it spells personalized with an “s” — isn’t it really Enterprise Learning Systems Considered Harmful to Learning?

The gift keeps on giving: there’s apparently a del.icio.us tag for PLEs. I know because my article was tagged by someone under it. And among those articles are ones that deal with these questions of how loose the PLE should be, ala Blackall.

(Why so few American representatives, I wonder? It’s all Canada, England, New Zealand, and Australia…)

I don’t think any of these ideas are new, really; it’s more that they’ve been refined during the long dark reign of the LMS. Looking at the network of people I’ve stumbled into I can see that they’ve been pushing these ideas outside the mainstream for some time too.

But I can’t help but feel that something is starting to happen here, when so many unrelated people are coming to the same conclusion. The very power of blogs to do what we see here — to organize people and refine ideas, to propel thought forward, to get things done — is what has revealed the LMS model to be such a cruel joke. So it’s not surprising, perhaps, that as blogging becomes ubiquitous these ideas, once considered digital utopianism, now can be expressed in very real and practical terms.

And even where the ideas are old, they now relate to a trailing-edge frame of reference — or soon will.

Over the next couple of days I’ll sift through my newly found goodies, and share what I find. I have a feeling it will be pretty extraordinary.