Quality of Input

In technosolutionist circles, the belief is that given the right algorithm we can make use of the massive amounts of information on the web to predict and solve problems. To the technosolutionist, the recent failure of advanced epidemic detecting tools to spot an ebola outbreak until a day after it had been announced by Guinea’s government through broadcast media should be a wake-up call. Foreign Policy gets it right:

On a panel I served on last week, we were asked to name what we thought was the greatest challenge to better understanding the world.  A representative of a government-funded agency stated that, in his program’s view, it was a need for better computer science tools to better extract patterns from data. That’s a worthwhile goal, but not if the data set is incomplete. While there is certainly great need for better data tools, even if one could perfectly extract every piece of information from theNew York Times each day, it would likely not yield a picture of the emerging Ebola outbreak any more detailed than what American government officials already have. Instead, what we truly need is better, more local data (and expanded tools that can translate and process that material) to allow us to more closely listen to and understand local communities.

You see this all over the place. There’s belief that the information is out there, we only need the tools to parse it. If you’re a twenty-something Silicon Valley native at a tech startup I’m sure it can feel like there’s more than enough information in some database or another to answer any question of importance to you.

For most of the world this is not the case, and you don’t even need to go to Guinea to find examples of this (I could show you this problem in my own organization, or the classroom of your choice).

I know Big Data is all so very exciting, but it would be great if we also took the collaborative/cooperative tools that have been stagnating and made them cheaper, better, more open, and less oppressive. It would be great if we poured some money into hiring more people whose job is to cultivate public networks, and if we’d pay people to translate materials from other places rather than just assuming strapping smartwatches to everybody will take care of it. It’d be nice to pull people into the process who specialized at pulling others in. At some point algorithms will matter most, but right now it’s the quality, quantity, and representativeness of input that represent the real roadblocks to better networked problem-solving

 


Maybe the reason people don’t use LMS collaboration tools is the tools are not collaborative

The video below, entitled “Why the Blackboard Wiki Is Not a Wiki”, shows how amazingly boneheaded Blackboard’s wiki tool design is. At the heart of the boneheadedness? The core idea of a wiki is that collaboration happens by way of

  • making things quick, and
  • seeing error and omission as community-creating opportunities, and
  • encouraging iteration

Blackboard, on the other hand, sees the job of a wiki as providing an interface to build finished pages. The Blackboard wiki is not distinguishable from the CMS your school uses to edit its website, except for the fact that it’s more poorly designed.

In other words, the collaboration tool is not a collaboration tool at all. It’s certainly not a wiki — a wiki, by definition, has page-creating links and other features that encourage organic growth. And as I demonstrate in the video there are no page-creating links, and everything possible is done to swat down the idea of emergent structure.

A report is out this week from EDUCAUSE on the LMS saying that the least liked and least used elements are the collaboration tools. What collaboration tools are those, exactly? I look at Blackboard and I’m not sure if there is a collaboration tool in there that wouldn’t seem right at home in 1999. Instead of thoughts about flow, we get buttons. Instead of buzz, we get stability.

bb_toolbar

Behind the scenes, it’s a big mess of HTML — no wiki markup, Markdown. No drag and drop.

wiki_markup

 

My guess is it’s called a “wiki” for only one reason — they have to check off the RFP box that says Blackboard has wikis. No one who has ever used a wiki has worked on this software, I would guess. I’m not even sure anyone even tried to collaborate in this space — actually collaborate, that is, not use it as a book report publisher.  I can’t imagine anyone typing about anything they care about into these boxes and thinking, this feels really cool.

The report finds people would like to collaborate in the LMS more, but don’t use the tools, and the recommendation is to provide training:

For academic technology personnel, the findings suggest the importance of focusing faculty and student training and support on LMS features that support collaboration and student engagement. Many of the underused LMS features (e.g., those that involve collaboration) have the potential to enhance student learning and engagement.

I agree with much of the report, but at least as concerns the collaboration tools the majority of the American market is stuck with, nothing could be further from the truth. My guess is that introduction to tools like this, called “wikis”, could only do harm; it’s like giving someone a 1997 HTML editor and FTP client and telling them this is “blogging”.

It may be that LMS’s can get wikis right, and if so they should (Canvas’s wiki gets much muuch closer to the mark, for example). And at the point it’s actually a collaboration tool, I’ll let faculty know it’s there. Until then the real solution is better integration with outside firms who understand collaboration is not the same as multi-user publishing.

 


Course Management Workflow in Federated Wiki

One of the persistent issues in using a networked learning approach to things is the workflow. I’m a couple weeks into using Federated Wiki in my edtech course, and I thought I’d share my methods.

Visualizing Participation

Currently, I load up students into the neighborhood by clicking on group pages, and then hit the Recent Changes link to see what’s going on. When you do this, you start to see the beauty of the gradient chiclet design.

chiclets

I can look at this an immediately see some patterns. First, the student with the blue chiclet (left corner gradient) has made a pass through a bunch of articles. I hover over the chiclet to get the student’s site ID — here it’s jeju.

I can tell a couple things about jeju. Today she (presumably) read through her group’s other materials and forked them to her site. Then she wrote four articles of her own.

How do I know she wrote four articles of her own? Because there are four where she is the first (and only author). How do I know that she forked some from others? Because there’s a bunch of articles where she is the latest author of a series of authors that started with someone else (now if there was a lot of fast back and forth, we wouldn’t know for sure that she didn’t start the article and then fork later efforts, but for this class that’s not an issue yet).

chiclet3

 

How do we know it’s forked within her group? It’s a guess, but you see how the stuff that she is forking tends to have the same set of authors? I’m guessing those are members of her seven-person group. A quick hover confirms this.

This is good. What I’ve asked students to do in the class is to use forking kind of like a “like” button. You see something you like, you pull it into your space. Here’s the rules I’ve given to my class:

  • Your site has to contain the work of others on it. If you’re picky about what is on your site, look around for good stuff or edit questionable stuff.
  • If you haven’t forked anything to your site we’re going to assume you haven’t read your classmates work, but also
  • Expect to be called on to summarize stuff you’ve forked to your site in class — don’t fork blindly.

We don’t know if jeju is forking blindly, but she is doing everything else right. As a bonus, we can call on her in class, knowing she’s read and forked the “Framing Essential Questions” and asked what she liked about that piece by kemc (another hover).

I’m curious whether the forking contains any editing. I told them they don’t need to edit for the moment, but I’m hoping some can’t help themselves. Let’s click one of these pages and take a look at the journal — say, How To Take Notes:

notes

So this ends up being a good example. On the left you see the original poster’s version, and on the right the latest version. Now, I actually added the comment and video-fied the video here for the student, but you can see below my yellow comment another student added a comment on this. Again, in the super neat way this works not only does the comment go on the site of the person commenting, but the page gets stored there too, in the form it was when they commenting.

I’m interested in who put that comment there, so I click on the fork associated with it and find it’s kemc. I’ll have to remember to call on him next class and thank him for leaving this comment which ties together a notetaking method with some principles in instructional design.

kemc

I’ve already commented on this one, but I should find new ones and encourage the students.

Commenting is a work in progress, but there’s already a number of features that support the larger workflow. For instance, a persistent thing that teachers want to do is find the new stuff by students that they haven’t commented on yet.

That’s easy here, because when I comment, it forks the page to my site. To find the sites I haven’t commented on, I just look for pages not yet in my site.  Below is a list of sites with my icon, hence I’ve commented/forked those:

have_reviewed

Further up the crawl, though are a bunch without my icon — I need to look at these:

todo

Note that a side of effect of this is that in each instance I am capturing a snapshot of the student material at the time I provided the formative feedback, which may be important if you have policies that are agressive about preservation of digital assignments.

The whole feel of the tool reminds me of Tufte’s take on interfaces (unfortunately rarely seen): at a glance, the interface allows you to spot trends, but when you look closer, all the supporting detail is there.

Speaking of Tufte, the newer version of federated wiki contains an additional sparkline visualization (a Tufte invention) for all pages in the lineup (currently loaded pages). This allows you to see at a glance the distribution of activity on a page (or pages) over time.

b94c7c8c-41d3-11e4-99a3-74a624a56622

Since it’s easy to shift-click and load up multiple pages quickly from Recent Changes, you could choose the pages you are interesting in comparing, then launch this tool on them. More on that when I get it installed.


Convivial Tools and Connected Courses

Excellent, must-read post from the Terry Elliot in the Connected Courses conversation which pulls in ideas of Christopher Alexanders’ System A (the organic, generative) and System B (the industrial, dead). Key grafs (for me at least):

I have a lot of questions about whether any of the web-based tools we are using actually fit the mold of System A. I don’t often feel those spaces as convivial and natural. Behind the artifice of interface lay the reality of code. Is that structure humane? Is it open, sustainable, and regenerative? Does it feel good? Does the whole idea behind code generate System A or System B? I really don’t know.

What I do know is that I get the very distinct feeling that certain systems I use are not convivial. Google+, Facebook, WordPress, Twitter while full of humans, feel closed, feel like templates to be filled in not spaces to be lived in. Hence, the need for outsiders more than ever to raise the question especially in this week of connected courses where we are talking about the why of why.

As readers know, I’ve been on an Alexander kick lately. And it’s less that Alexander led me to these sorts of questions than questions that have been disturbing me have led me to Alexander. So I probably have a less useful perspective than someone that comes to this with a wealth of Alexandrian insight.

“Templates to be filled, not places to be lived in.” Hmmm.

Maybe some of this unavoidable. But I wonder in particular if some of it is the perils of StreamMode, that tendency to conceptualize all of out digital life as a stream of events and statements reacting to other events and statements in a never-ending crawl. The problem with StreamMode is that the structures that make StreamMode coherent are past conversations and concepts newbies don’t have access to. StreamMode also relies heavily on personalities, and hence, popularity.

Look at this blog post, for instance. You want to know what StreamMode is? Do I link to to a definition? No, not hardly. I link you to an older piece that kinda-sorta defines the term in a context that involves a bunch of people and posts you don’t know about. How humane is that?

StateMode is a little different. StateMode is like a wiki — at any given point in time the wiki represents the total documented understanding of the community. The voice that develops is generic or semi-generic, and aims to be architecture, not utterance. If you want the feeling of StateMode, go to a place like TV Tropes. Look past the ads and you’ll find the site invites you into the community as living architecture instead of stream. New articles form as ways to make older articles more meaningful, or understandable. The process is recursive, not episodic.

The problem is that StreamMode builds community at the expense of coherence, and StateMode builds coherence at the expense of community.

I think this may be one of those irreducible conundrums, but I also think over the past 10 years we have veered too much into StreamMode, which gives us not that timeless sense but an overwhelming wave of personality pinging off of personality.

Ages ago on the Internet you used to stuble onto weird and wonderful mini-sites, like secret gardens found in the middle of the woods. Now we find streams of conversation, endlessly repeating, pushing us to live in a narrative that is not ours. The expressive nature of the web is to be treasured, but I think we’ve lost something.


The LMS in Historical Perspective

Is it ironic that I’m posting on this topic partially to avoid the mundane process of setting up Blackboard trainings for a bit longer? Probably not.

In any case, Jared Stein has a post up about the LMS and his personal perspective on why it turned out the way it did, and the crux of the narrative is that the LMS won out initially because frankly there was a bunch of stuff that the web was not making easy at the time. And the primary force for adoption of the LMS was not administration, but faculty, who really wanted management tools.

I’m too young on this side of higher education to confirm or deny that account as the ground truth, but it jives with my experience as well. My first encounter with Blackboard was on a educational simulation project we were looking to sell to Old Dominion University in 2000/ early 2001. Cognitive Arts wanted to sell a kickass (technical term) learning sim we had made to Old Dominion to run, something we had spent several millions of dollars developing, and which had had great results (I’m trying to remember if it was our macroecon product or our Java 101 product). Anyway, word came back from the sales team — they are interested, but they want it to run in Blackboard. Since I was usually the “Somebody figure out what X is” person on the team, it fell to me to figure out if we could integrate it.

I ultimately learned we could wrap our web hosted software in a frame (it was 2000, deal with it) and exchange some very rudimentary data with Bb. I think it might have just been a completion flag, and I forget how we hacked it.(Other places we had wrapped things as big AICC objects, but I’m not sure that was the method here). Ultimately the whole deal turned out to be a sales mirage, so I thanked sales for another ill-spent 65-hour week chasing after phantom commissions for them, and called it a day.

But here’s the thing. As I remember, it wasn’t an institutional rule that it had to fit into Bb. It was a faculty concern that they pushed up to us. Faculty thought our stuff was cool, but they didn’t want to be in two places. They wanted to manage it from Blackboard, and wanted it to be available to the students from Blackboard so that the course felt coherent rather than two seperate, unconnected bits.

My second exposure to the LMS was getting to know the guys who built Prometheus, which at the time was billed as the first “community source” LMS. (Actually, now that I think of it, it was my first LMS exposure, since I remember meeting them in Washington D.C. when Bill Clinton was still President).

I had co-built the enrollment and user managment functions in our Cognitive Arts’ home-brew LMS (let’s admit it, if you enroll users, its an LMS whether it’s a syndication hub or a simulation). And when I looked under the hood of Prometheus (which was written in ColdFusion, the same language as our product) I went right over to my boss and argued we port our stuff to Prometheus and ditch our home-brew. It was just elegant and extensible. It looked beautiful. It was ridiculously easy to write extensions to it.

Here was a platform that encouraged you to build on top of it. And a community that was truly engaged with the possibilities extending it.

Of course, Prometheus was eventually bought by Blackboard. If you’ve heard of Building Blocks in Blackboard, it’s basically a port of what Prometheus was doing with extensibility, but done in a way that makes it unattractive to end-users to build in it. So yay.

A number of years later I asked a person I knew who worked at Prometheus why Prometheus failed. Did Blackboard crush them?

His answer was interesting. No, it wasn’t Blackboard at all. It was the educational institutions. With the slow, resource-intensive and state-mandated RFP processes, the interminable faculty commitees,  and the way that even after the deal was signed the institution would delay payment and implementation as long as possible (or suddenly throw it into an unanticipated ‘final review’) it was just not possible to grow a stable business. The process institutions followed was supposed to ensure equitable access to contracts, but what it did was made it impossible for any company not sitting on a pile of cash to stay in business. (I’m extrapolating a bit here, but not much).

So ironically, if you want to know what built the world of the One True LMS, look not at capitalism, but anti-corruption law, faculty governance, and state budgeting. Fun, huh? Counter to most of the rest of my world-view, but from what I’ve seen on the inside, entirely true.

When I got to Keene State in 2004, Blackboard was entrenched, but the stories I heard were really the same. There had been a faculty push to get something in place, and the Blackboard advocates had won. But what we started finding as 2004 rolled into 2005 and 2006 was that the external web was accelerating making all sorts of open connection possible, all sorts of reuse possible, and Blackboard was supporting none of it. They decided they had solved the LMS problem, once and for all, and started to look at more lucrative aspects of university business.

The standard joke about Blackboard by 2007 was that they were spending millions of dollars trying to get your university’s keycard production business (actually true) but couldn’t figure out a way for you to add a simple YouTube video without writing a javascript-based hack.

By 2008, when Blackboard announced, hey, we’re so Web 2.0, I was writing rather high-minded screeds like this:

[W]hile Blackboard was busy trying to leverage their foothold in the University to get into the business of dining hall management, video surveillance, and door access control, this little thing called Web 2.0 happened. And suddenly the technology Blackboard had for learning began to look — well, old. Junky. Very 1999.

So while Bb spent their efforts trying to become the single sign-on point for your institution, professors, frustrated with the kludginess of the actual *learning* part of Bb’s suite, started looking elsewhere for solutions.

Their first discovery was that they could do everything they were doing in Blackboard for free, and much more easily.

But the second discovery was the kicker. These Web 2.0 tools they adopted encouraged them to share their stuff with the world, instead of locking it away in a password protected course. And suddenly, they got a taste of open education. And it didn’t stop there. The tools they adopted had a true web DNA, and played well with other tools in a loosely coupled mode. So suddenly, they got a taste of what it was like to build your own custom learning environment.

The tone is a bit overstated and obnoxious, but I stand by the 2008 analysis. Somewhere in the mid-aughts, the customer of Blackboard moved from being the faculty member (who they had aggressively courted initially) to the administration. And, as Scott Leslie points out, incumbency’s a bitch. They became a barrier to serious, important movements — the social web, open education. By the time I was working for the OpenCourseWare Consortium it had passed ridiculousness into something akin to a crime against humanity. Again and again I’d talk to people who wanted to open up their instructional materials to the world only to learn that the company they had paid a quarter of a million dollars to host those materials had no way to let them do that.

[Side note: Insert argument here about whether all classes really need "educational materials" which is about as intelligent as arguing about whether all classes need students to buy novels.]

I’m interested where Jared takes the history from there. But what he has described so far fits my recollection. We fought the LMS, the LMS won, and then the LMS just sat there for a number of years asking us if we would like to upgrade to the alumni fundraising product. What happened next was… well, back to setting up Blackboard training [sigh]. But I’ll reply with more when Part II of Jared’s series comes out.


Reply to D’Arcy

D’Arcy Norman’s blog has a great policy which I may go back to soon — you can’t comment on his blog, only trackback to it. So here’s me commenting on his blog.

In a recent post, D’Arcy expands on this idea, among others:

Any eLearning tool, no matter how openly designed, will eventually become indistinguishable from a Learning Management System once a threshold of supported use-cases has been reached.

This is true, and it is something we need to come to terms with. At a certain point, it’s not UMW Blogs vs. the LMS anymore. It’s the UMW Blogs LMS vs. other LMS’s. And as you try to support more use cases — if, for example, you suddenly needed to run all your courses on nothing but the syndication hub architecture — you will find that the complexity and bloat creeps right back in.

I’ll add that I’ve been looking at these two pictures a bit:

h.OuterStripNewmarket1990sweb

And someone will tell me it’s the wrong analogy I suppose, but it reminds me that we talk a lot about the problem on the left (call it Facebook, the LMS, whatever) and not enough about the problem on the right. A world where each class defines its own architecture without regard to any other class starts to look like the world on the right. Things like WordPress work best when they are approached (at least partially) from an enterprise-level view, as unfortunate as that may be.

Broadly, I think what is happening with both WordPress learning systems and LMS’s is both are moving into federated designs. WordPress is working there from the bottom up, via syndication hubs and the like. The LMS, on the other hand,  is decentralizing — slowly becoming an authentication, communication, and assessement hub for outside technologies. I personally think the homegrown systems are far more consistent at this point with how I like to organize classes, but it would be hard to deny the sea-change in LMS focus over the past couple years.

UPDATE: Phil Hill also comments.


Learning Design Pattern #121: You’re Doing It Wrong

youre-doing-it-wrong1

Had a great lunch today with Michael Berman in Portland, and boy am I glad I got down there. We talked about my recent fascination with the idea of Learning Design Patterns, and more broadly with agile methods in learning design. I mentioned that one thing that was a struggle was getting the pattern at the right level. The trick with patterns is they must be concrete enough that you can “think with them” but broad enough that they can generate unexpected solutions.

Michael, who worked a bit with design patterns in the 1990s, came up with a pattern that I think is at just the right level. He called it something along the lines of “Doing it Wrong”, pulling together the Trolling exercises of ds106, the public speaking exercises where you ask students to give a poor speech (e.g. mumble, use horrible powerpoint slides, don’t make eye contact, give a series of points that don’t relate), and the stats exercises I used to do where I told students to create a “biased data visualization” through using weird cut points, truncated Y-axes, strange groupings (lumping together people “shot OR killed”), and uncontrolled data.

At the moment, I think this is just the right level for a learning design pattern. There’s some underlying cognitive logic here about how we deconstruct experience. There’s an ability to match the pattern with higher order patterns about course design (what we might call the “edges”) and lower order patterns on the level of a single class session. It’s bigger and more generative than an Assignment Bank assignment, but smaller than a methodological category.

You might also have a design called “Formal Commitment”, where you push students to commit to an answer before discussion. You see this in a number of places such as Peer Instruction and some exercises from Brookfield’s Structured Discussion. There’s  a lot of evidence that students need to commit to a Theory of the Moment, even if it is only temporarily, so that they can more rigorously explore a question. Students who don’t internalize a theory or prediction can’t see when that theory fails. This pattern might plug into some higher order patterns about targeting preconceptions or some lower order ones about Role Play.

Why does talking in this way matter? Because there’s something really special about well-delineated ideas expressed at that level. If they truly plug into something fundamental about cognition, we can learn things about this group of assignments as a class. What are some issues around the set-up of “You’re doing it wrong” assignments? What are some challenges of assessment?

If it’s a particularly good pattern, we’d also learn how it relates to certain environments — in learning design cases that environment might be a discipline, or the particular talents of your students.

Most importantly it’s at that crucial generative level. When we think in terms of activities, it’s too easy to get lost in the detail and lose track of what we are trying to accomplish. Theory, on the other hand, understands the point, but is too abstract to lead directly towards solutions. Patterns represent a sort embedded theory that provides the coherence and rigor of the theoretical while generating the warmth and resonance of the particular.

The larger goal is to create a middle ground between the deadness of current Big Design approaches to instructional design and the anarchy of ignoring design altogether. Because Big Design gives you this:

Markham-suburbs_aerial-edit2

And design anarchy gives you this:

h.OuterStripNewmarket1990sweb

And what you really want is this:

nh-gazebo

Not to say a church and gazebo, mind you. But you’re looking a three centuries of architecture above in the Keene square, built by dozens of different people with no sense of urban theory and, for the most part, no architecture training. And yet it looks organic, it feels whole, like a single piece, and it functions better than most anything you’ll find anywhere. Why? Because the patterns of the New England town were internalized in such a way that you could build with the coherence of theory and predictibility of large-scale process without having to use theory or large-scale process.

I don’t know if design patterns is what gets us to this in learning design. I don’t know if you could ever reliably produce an education that works as well or feels as human as the New England town square. But it at least gets us away from the “10 tips for teaching” nonsense while avoiding theory. So let’s give it a go, right?


Follow

Get every new post delivered to your Inbox.

Join 134 other followers