Quality of Input

In technosolutionist circles, the belief is that given the right algorithm we can make use of the massive amounts of information on the web to predict and solve problems. To the technosolutionist, the recent failure of advanced epidemic detecting tools to spot an ebola outbreak until a day after it had been announced by Guinea’s government through broadcast media should be a wake-up call. Foreign Policy gets it right:

On a panel I served on last week, we were asked to name what we thought was the greatest challenge to better understanding the world.  A representative of a government-funded agency stated that, in his program’s view, it was a need for better computer science tools to better extract patterns from data. That’s a worthwhile goal, but not if the data set is incomplete. While there is certainly great need for better data tools, even if one could perfectly extract every piece of information from theNew York Times each day, it would likely not yield a picture of the emerging Ebola outbreak any more detailed than what American government officials already have. Instead, what we truly need is better, more local data (and expanded tools that can translate and process that material) to allow us to more closely listen to and understand local communities.

You see this all over the place. There’s belief that the information is out there, we only need the tools to parse it. If you’re a twenty-something Silicon Valley native at a tech startup I’m sure it can feel like there’s more than enough information in some database or another to answer any question of importance to you.

For most of the world this is not the case, and you don’t even need to go to Guinea to find examples of this (I could show you this problem in my own organization, or the classroom of your choice).

I know Big Data is all so very exciting, but it would be great if we also took the collaborative/cooperative tools that have been stagnating and made them cheaper, better, more open, and less oppressive. It would be great if we poured some money into hiring more people whose job is to cultivate public networks, and if we’d pay people to translate materials from other places rather than just assuming strapping smartwatches to everybody will take care of it. It’d be nice to pull people into the process who specialized at pulling others in. At some point algorithms will matter most, but right now it’s the quality, quantity, and representativeness of input that represent the real roadblocks to better networked problem-solving


Maybe the reason people don’t use LMS collaboration tools is the tools are not collaborative

The video below, entitled “Why the Blackboard Wiki Is Not a Wiki”, shows how amazingly boneheaded Blackboard’s wiki tool design is. At the heart of the boneheadedness? The core idea of a wiki is that collaboration happens by way of

  • making things quick, and
  • seeing error and omission as community-creating opportunities, and
  • encouraging iteration

Blackboard, on the other hand, sees the job of a wiki as providing an interface to build finished pages. The Blackboard wiki is not distinguishable from the CMS your school uses to edit its website, except for the fact that it’s more poorly designed.

In other words, the collaboration tool is not a collaboration tool at all. It’s certainly not a wiki — a wiki, by definition, has page-creating links and other features that encourage organic growth. And as I demonstrate in the video there are no page-creating links, and everything possible is done to swat down the idea of emergent structure.

A report is out this week from EDUCAUSE on the LMS saying that the least liked and least used elements are the collaboration tools. What collaboration tools are those, exactly? I look at Blackboard and I’m not sure if there is a collaboration tool in there that wouldn’t seem right at home in 1999. Instead of thoughts about flow, we get buttons. Instead of buzz, we get stability.


Behind the scenes, it’s a big mess of HTML — no wiki markup, Markdown. No drag and drop.



My guess is it’s called a “wiki” for only one reason — they have to check off the RFP box that says Blackboard has wikis. No one who has ever used a wiki has worked on this software, I would guess. I’m not even sure anyone even tried to collaborate in this space — actually collaborate, that is, not use it as a book report publisher.  I can’t imagine anyone typing about anything they care about into these boxes and thinking, this feels really cool.

The report finds people would like to collaborate in the LMS more, but don’t use the tools, and the recommendation is to provide training:

For academic technology personnel, the findings suggest the importance of focusing faculty and student training and support on LMS features that support collaboration and student engagement. Many of the underused LMS features (e.g., those that involve collaboration) have the potential to enhance student learning and engagement.

I agree with much of the report, but at least as concerns the collaboration tools the majority of the American market is stuck with, nothing could be further from the truth. My guess is that introduction to tools like this, called “wikis”, could only do harm; it’s like giving someone a 1997 HTML editor and FTP client and telling them this is “blogging”.

It may be that LMS’s can get wikis right, and if so they should (Canvas’s wiki gets much muuch closer to the mark, for example). And at the point it’s actually a collaboration tool, I’ll let faculty know it’s there. Until then the real solution is better integration with outside firms who understand collaboration is not the same as multi-user publishing.


Course Management Workflow in Federated Wiki

One of the persistent issues in using a networked learning approach to things is the workflow. I’m a couple weeks into using Federated Wiki in my edtech course, and I thought I’d share my methods.

Visualizing Participation

Currently, I load up students into the neighborhood by clicking on group pages, and then hit the Recent Changes link to see what’s going on. When you do this, you start to see the beauty of the gradient chiclet design.


I can look at this an immediately see some patterns. First, the student with the blue chiclet (left corner gradient) has made a pass through a bunch of articles. I hover over the chiclet to get the student’s site ID — here it’s jeju.

I can tell a couple things about jeju. Today she (presumably) read through her group’s other materials and forked them to her site. Then she wrote four articles of her own.

How do I know she wrote four articles of her own? Because there are four where she is the first (and only author). How do I know that she forked some from others? Because there’s a bunch of articles where she is the latest author of a series of authors that started with someone else (now if there was a lot of fast back and forth, we wouldn’t know for sure that she didn’t start the article and then fork later efforts, but for this class that’s not an issue yet).



How do we know it’s forked within her group? It’s a guess, but you see how the stuff that she is forking tends to have the same set of authors? I’m guessing those are members of her seven-person group. A quick hover confirms this.

This is good. What I’ve asked students to do in the class is to use forking kind of like a “like” button. You see something you like, you pull it into your space. Here’s the rules I’ve given to my class:

  • Your site has to contain the work of others on it. If you’re picky about what is on your site, look around for good stuff or edit questionable stuff.
  • If you haven’t forked anything to your site we’re going to assume you haven’t read your classmates work, but also
  • Expect to be called on to summarize stuff you’ve forked to your site in class — don’t fork blindly.

We don’t know if jeju is forking blindly, but she is doing everything else right. As a bonus, we can call on her in class, knowing she’s read and forked the “Framing Essential Questions” and asked what she liked about that piece by kemc (another hover).

I’m curious whether the forking contains any editing. I told them they don’t need to edit for the moment, but I’m hoping some can’t help themselves. Let’s click one of these pages and take a look at the journal — say, How To Take Notes:


So this ends up being a good example. On the left you see the original poster’s version, and on the right the latest version. Now, I actually added the comment and video-fied the video here for the student, but you can see below my yellow comment another student added a comment on this. Again, in the super neat way this works not only does the comment go on the site of the person commenting, but the page gets stored there too, in the form it was when they commenting.

I’m interested in who put that comment there, so I click on the fork associated with it and find it’s kemc. I’ll have to remember to call on him next class and thank him for leaving this comment which ties together a notetaking method with some principles in instructional design.


I’ve already commented on this one, but I should find new ones and encourage the students.

Commenting is a work in progress, but there’s already a number of features that support the larger workflow. For instance, a persistent thing that teachers want to do is find the new stuff by students that they haven’t commented on yet.

That’s easy here, because when I comment, it forks the page to my site. To find the sites I haven’t commented on, I just look for pages not yet in my site.  Below is a list of sites with my icon, hence I’ve commented/forked those:


Further up the crawl, though are a bunch without my icon — I need to look at these:


Note that a side of effect of this is that in each instance I am capturing a snapshot of the student material at the time I provided the formative feedback, which may be important if you have policies that are agressive about preservation of digital assignments.

The whole feel of the tool reminds me of Tufte’s take on interfaces (unfortunately rarely seen): at a glance, the interface allows you to spot trends, but when you look closer, all the supporting detail is there.

Speaking of Tufte, the newer version of federated wiki contains an additional sparkline visualization (a Tufte invention) for all pages in the lineup (currently loaded pages). This allows you to see at a glance the distribution of activity on a page (or pages) over time.


Since it’s easy to shift-click and load up multiple pages quickly from Recent Changes, you could choose the pages you are interesting in comparing, then launch this tool on them. More on that when I get it installed.

The LMS in Historical Perspective

Is it ironic that I’m posting on this topic partially to avoid the mundane process of setting up Blackboard trainings for a bit longer? Probably not.

In any case, Jared Stein has a post up about the LMS and his personal perspective on why it turned out the way it did, and the crux of the narrative is that the LMS won out initially because frankly there was a bunch of stuff that the web was not making easy at the time. And the primary force for adoption of the LMS was not administration, but faculty, who really wanted management tools.

I’m too young on this side of higher education to confirm or deny that account as the ground truth, but it jives with my experience as well. My first encounter with Blackboard was on a educational simulation project we were looking to sell to Old Dominion University in 2000/ early 2001. Cognitive Arts wanted to sell a kickass (technical term) learning sim we had made to Old Dominion to run, something we had spent several millions of dollars developing, and which had had great results (I’m trying to remember if it was our macroecon product or our Java 101 product). Anyway, word came back from the sales team — they are interested, but they want it to run in Blackboard. Since I was usually the “Somebody figure out what X is” person on the team, it fell to me to figure out if we could integrate it.

I ultimately learned we could wrap our web hosted software in a frame (it was 2000, deal with it) and exchange some very rudimentary data with Bb. I think it might have just been a completion flag, and I forget how we hacked it.(Other places we had wrapped things as big AICC objects, but I’m not sure that was the method here). Ultimately the whole deal turned out to be a sales mirage, so I thanked sales for another ill-spent 65-hour week chasing after phantom commissions for them, and called it a day.

But here’s the thing. As I remember, it wasn’t an institutional rule that it had to fit into Bb. It was a faculty concern that they pushed up to us. Faculty thought our stuff was cool, but they didn’t want to be in two places. They wanted to manage it from Blackboard, and wanted it to be available to the students from Blackboard so that the course felt coherent rather than two seperate, unconnected bits.

My second exposure to the LMS was getting to know the guys who built Prometheus, which at the time was billed as the first “community source” LMS. (Actually, now that I think of it, it was my first LMS exposure, since I remember meeting them in Washington D.C. when Bill Clinton was still President).

I had co-built the enrollment and user managment functions in our Cognitive Arts’ home-brew LMS (let’s admit it, if you enroll users, its an LMS whether it’s a syndication hub or a simulation). And when I looked under the hood of Prometheus (which was written in ColdFusion, the same language as our product) I went right over to my boss and argued we port our stuff to Prometheus and ditch our home-brew. It was just elegant and extensible. It looked beautiful. It was ridiculously easy to write extensions to it.

Here was a platform that encouraged you to build on top of it. And a community that was truly engaged with the possibilities extending it.

Of course, Prometheus was eventually bought by Blackboard. If you’ve heard of Building Blocks in Blackboard, it’s basically a port of what Prometheus was doing with extensibility, but done in a way that makes it unattractive to end-users to build in it. So yay.

A number of years later I asked a person I knew who worked at Prometheus why Prometheus failed. Did Blackboard crush them?

His answer was interesting. No, it wasn’t Blackboard at all. It was the educational institutions. With the slow, resource-intensive and state-mandated RFP processes, the interminable faculty commitees,  and the way that even after the deal was signed the institution would delay payment and implementation as long as possible (or suddenly throw it into an unanticipated ‘final review’) it was just not possible to grow a stable business. The process institutions followed was supposed to ensure equitable access to contracts, but what it did was made it impossible for any company not sitting on a pile of cash to stay in business. (I’m extrapolating a bit here, but not much).

So ironically, if you want to know what built the world of the One True LMS, look not at capitalism, but anti-corruption law, faculty governance, and state budgeting. Fun, huh? Counter to most of the rest of my world-view, but from what I’ve seen on the inside, entirely true.

When I got to Keene State in 2004, Blackboard was entrenched, but the stories I heard were really the same. There had been a faculty push to get something in place, and the Blackboard advocates had won. But what we started finding as 2004 rolled into 2005 and 2006 was that the external web was accelerating making all sorts of open connection possible, all sorts of reuse possible, and Blackboard was supporting none of it. They decided they had solved the LMS problem, once and for all, and started to look at more lucrative aspects of university business.

The standard joke about Blackboard by 2007 was that they were spending millions of dollars trying to get your university’s keycard production business (actually true) but couldn’t figure out a way for you to add a simple YouTube video without writing a javascript-based hack.

By 2008, when Blackboard announced, hey, we’re so Web 2.0, I was writing rather high-minded screeds like this:

[W]hile Blackboard was busy trying to leverage their foothold in the University to get into the business of dining hall management, video surveillance, and door access control, this little thing called Web 2.0 happened. And suddenly the technology Blackboard had for learning began to look — well, old. Junky. Very 1999.

So while Bb spent their efforts trying to become the single sign-on point for your institution, professors, frustrated with the kludginess of the actual *learning* part of Bb’s suite, started looking elsewhere for solutions.

Their first discovery was that they could do everything they were doing in Blackboard for free, and much more easily.

But the second discovery was the kicker. These Web 2.0 tools they adopted encouraged them to share their stuff with the world, instead of locking it away in a password protected course. And suddenly, they got a taste of open education. And it didn’t stop there. The tools they adopted had a true web DNA, and played well with other tools in a loosely coupled mode. So suddenly, they got a taste of what it was like to build your own custom learning environment.

The tone is a bit overstated and obnoxious, but I stand by the 2008 analysis. Somewhere in the mid-aughts, the customer of Blackboard moved from being the faculty member (who they had aggressively courted initially) to the administration. And, as Scott Leslie points out, incumbency’s a bitch. They became a barrier to serious, important movements — the social web, open education. By the time I was working for the OpenCourseWare Consortium it had passed ridiculousness into something akin to a crime against humanity. Again and again I’d talk to people who wanted to open up their instructional materials to the world only to learn that the company they had paid a quarter of a million dollars to host those materials had no way to let them do that.

[Side note: Insert argument here about whether all classes really need "educational materials" which is about as intelligent as arguing about whether all classes need students to buy novels.]

I’m interested where Jared takes the history from there. But what he has described so far fits my recollection. We fought the LMS, the LMS won, and then the LMS just sat there for a number of years asking us if we would like to upgrade to the alumni fundraising product. What happened next was… well, back to setting up Blackboard training [sigh]. But I’ll reply with more when Part II of Jared’s series comes out.

Reply to D’Arcy

D’Arcy Norman’s blog has a great policy which I may go back to soon — you can’t comment on his blog, only trackback to it. So here’s me commenting on his blog.

In a recent post, D’Arcy expands on this idea, among others:

Any eLearning tool, no matter how openly designed, will eventually become indistinguishable from a Learning Management System once a threshold of supported use-cases has been reached.

This is true, and it is something we need to come to terms with. At a certain point, it’s not UMW Blogs vs. the LMS anymore. It’s the UMW Blogs LMS vs. other LMS’s. And as you try to support more use cases — if, for example, you suddenly needed to run all your courses on nothing but the syndication hub architecture — you will find that the complexity and bloat creeps right back in.

I’ll add that I’ve been looking at these two pictures a bit:


And someone will tell me it’s the wrong analogy I suppose, but it reminds me that we talk a lot about the problem on the left (call it Facebook, the LMS, whatever) and not enough about the problem on the right. A world where each class defines its own architecture without regard to any other class starts to look like the world on the right. Things like WordPress work best when they are approached (at least partially) from an enterprise-level view, as unfortunate as that may be.

Broadly, I think what is happening with both WordPress learning systems and LMS’s is both are moving into federated designs. WordPress is working there from the bottom up, via syndication hubs and the like. The LMS, on the other hand,  is decentralizing — slowly becoming an authentication, communication, and assessement hub for outside technologies. I personally think the homegrown systems are far more consistent at this point with how I like to organize classes, but it would be hard to deny the sea-change in LMS focus over the past couple years.

UPDATE: Phil Hill also comments.

Learning Design Pattern #121: You’re Doing It Wrong


Had a great lunch today with Michael Berman in Portland, and boy am I glad I got down there. We talked about my recent fascination with the idea of Learning Design Patterns, and more broadly with agile methods in learning design. I mentioned that one thing that was a struggle was getting the pattern at the right level. The trick with patterns is they must be concrete enough that you can “think with them” but broad enough that they can generate unexpected solutions.

Michael, who worked a bit with design patterns in the 1990s, came up with a pattern that I think is at just the right level. He called it something along the lines of “Doing it Wrong”, pulling together the Trolling exercises of ds106, the public speaking exercises where you ask students to give a poor speech (e.g. mumble, use horrible powerpoint slides, don’t make eye contact, give a series of points that don’t relate), and the stats exercises I used to do where I told students to create a “biased data visualization” through using weird cut points, truncated Y-axes, strange groupings (lumping together people “shot OR killed”), and uncontrolled data.

At the moment, I think this is just the right level for a learning design pattern. There’s some underlying cognitive logic here about how we deconstruct experience. There’s an ability to match the pattern with higher order patterns about course design (what we might call the “edges”) and lower order patterns on the level of a single class session. It’s bigger and more generative than an Assignment Bank assignment, but smaller than a methodological category.

You might also have a design called “Formal Commitment”, where you push students to commit to an answer before discussion. You see this in a number of places such as Peer Instruction and some exercises from Brookfield’s Structured Discussion. There’s  a lot of evidence that students need to commit to a Theory of the Moment, even if it is only temporarily, so that they can more rigorously explore a question. Students who don’t internalize a theory or prediction can’t see when that theory fails. This pattern might plug into some higher order patterns about targeting preconceptions or some lower order ones about Role Play.

Why does talking in this way matter? Because there’s something really special about well-delineated ideas expressed at that level. If they truly plug into something fundamental about cognition, we can learn things about this group of assignments as a class. What are some issues around the set-up of “You’re doing it wrong” assignments? What are some challenges of assessment?

If it’s a particularly good pattern, we’d also learn how it relates to certain environments — in learning design cases that environment might be a discipline, or the particular talents of your students.

Most importantly it’s at that crucial generative level. When we think in terms of activities, it’s too easy to get lost in the detail and lose track of what we are trying to accomplish. Theory, on the other hand, understands the point, but is too abstract to lead directly towards solutions. Patterns represent a sort embedded theory that provides the coherence and rigor of the theoretical while generating the warmth and resonance of the particular.

The larger goal is to create a middle ground between the deadness of current Big Design approaches to instructional design and the anarchy of ignoring design altogether. Because Big Design gives you this:


And design anarchy gives you this:


And what you really want is this:


Not to say a church and gazebo, mind you. But you’re looking a three centuries of architecture above in the Keene square, built by dozens of different people with no sense of urban theory and, for the most part, no architecture training. And yet it looks organic, it feels whole, like a single piece, and it functions better than most anything you’ll find anywhere. Why? Because the patterns of the New England town were internalized in such a way that you could build with the coherence of theory and predictibility of large-scale process without having to use theory or large-scale process.

I don’t know if design patterns is what gets us to this in learning design. I don’t know if you could ever reliably produce an education that works as well or feels as human as the New England town square. But it at least gets us away from the “10 tips for teaching” nonsense while avoiding theory. So let’s give it a go, right?

Learning Design Patterns as an Alternative Model of Course Design

I’ve discovered Chris Alexander’s work on architecture, and I cannot read it without hearing every line as a statement on problems in course design.  Alexander approaches architecture not through top-down design or bottom-up chaos, but through generative constraints, that is, he begins with the environment and then runs through a “grammar” of building. The design emerges through these constraints the way a sentence arises from grammatical templates/rules (depending on your view of grammar).

I’m new to this, but the way it seems to work is you have a pattern such as “Light on Two Sides“. This pattern notes that people naturally gravitate to rooms with windows on at least two walls. This pattern is made possible by attending to some higher order patterns dealing with the “edge” of the house: Positive Outdoor Space, Wings of Light, and Long Thin House. If Light on Two Sides is impossible in spaces where you need it, you may need to revisit the higher order patterns.

This Light on Two Sides pattern when executed then becomes the higher order pattern to a number of even lower order patterns: Windows Overlooking Life, Deep Reveal, and Roof Layout. In other words the work proceeds in the way you might write a song or form the plot of a book. You write a series of notes that determines the key, which generates more melodies and suggests a bridge in the relative minor key. You add a drum beat which suggests for you the rhythmic structure of the bass. The bass suggests a keyboard hook. Layer by layer the song emerges as a living thing. Everything ends up original, but it is the process of putting one piece in place guides the next piece of work.

In pedagogy how would this work? I have a 150 year old book on teaching I recently read, and scattered through it are comments such as “Place Hard Work First” (e.g. students have limited metal effort to expend, put the activities requiring the most effort towards the front of the lesson). That might be placed alongside a pattern about Peer Learning — and if peer learning is effortful it would suggest the position of peer learning in your course. The use of peer learning might require a higher order pattern about the length of the class or the format of the furniture.

So if the patterns are the same, how do designs end up different? Because you start with different constraints. The same way the shape of the land or relationship to other houses will trigger certain solutions in Alexander’s pattern language:

“Above all, the shapes of the building must spring from the land, and buildings around, like a tree springing from a coppice — it fits perfectly, the moment of inception.”

Your students have certain backgrounds, various strengths. Your institution has certain facilities and your technology has certain affordances. Just as a limited number of grammar rules produce an infinite number of sentences based on the needs of the moment, so learning design patterns combined with the circumstances and aims of instruction can produce infinitely expressive learning designs.

What excites me about this is it is a way to combine research and practice without succumbing to an industrial paradigm. This isn’t “wing it on the whiteboard” or the class as the artistic expression of the instructor. This is a framework which is as rigorous in its own way as any ADDIE-fueled design monster. It’s premises can be challenged. It identifies right and wrong ways to go about things. It would evolve in reaction to new data. It is a distinct methodology which can be shown to produce either good or bad results. But, unlike many methodologies. it seems to me to work in the natural directon of our thought. And as Alexander points out it’s this pattern of a work reacting to itself and its environment that gives it the spark of life.

I am placing some quotes of Alexander’s here to give you the flavor of his thought. They come from random places with no original sourcing, so I paste them here without links. If you find them as powerful as I do the context will eventually present itself.

“In an organic environment, every place is unique, and the different places also cooperate, with no parts left over, to create a global whole – a whole which can be identified by everyone who is part of it.”

“In the past century, architecture has always been a minor science — if it has been a science at all. Present day architects who want to be scientific, try to incorporate the ideas of physics, psychology, anthropology in their work . . . in the hope of keeping in tune with the “scientific” times. I believe we are on the threshold of a new era, when this relation between architecture and the physical sciences may be reversed — when the proper understanding of the deep questions of space, as they are embodied in architecture will play a revolutionary role in the way we see the world and will do for the world view of the 21st and 22nd centuries, what physics did for the 19th and 20th.”

“Every building, every room, every garden is better when all the patterns which it needs are compressed as far as it is possible for them to be. The building will be cheaper; the meaning in it will be denser.”

“I’ll tell you a story. I was in India in 1961. I was living in a village most of the time. I studied that village, tried to understand what village life was all about. And I got back to Harvard, a few months later, and I got a letter from the government of [the town in India], saying ‘We’ve got to re-locate our village because of the dam construction. Would you like to build it?’. I think about 2000 people were being moved. And I thought about it. And then I was very sad. And I wrote back, and I said, ‘You know, I don’t know enough about how to do it. Because I don’t want to come in and simply build a village, because I don’t think that will make life. I know that the life has got to come from the people, as well as what’s going on physically, geometrically. My experience of living in the village is that I do not know enough about how to actually make that happen. And therefore I very very regretfully decline your kind offer.’ And I was actually chagrined beyond measure, that I had to give that reply. But it was honest, and in fact, it was because of that letter that I wrote A Pattern Language. Because, I thought and thought, and I said, ‘You know, this is crazy. What would I have to do, to put in people’s hands the thing with which they could do this, so that it would be like a real village and not like an architect’s fantasy?”

“We are searching for some kind of harmony between two intangibles: a form which we have not yet designed and a context which we cannot properly describe.”

“If you have a feeling-vision of the thing – a painting, a building, a garden, a piece of a neighborhood – as long as you’re very firmly anchored in your knowledge of that thing, and you can see it with your eyes closed, you can keep correcting your actions… It’s not a question of holding onto every little detail, but of holding onto the feeling.”

“From a sequence of these individual patterns, whole buildings with the character of nature will form themselves within your thoughts, as easily as sentences.”

“Nowadays, the process of growth and development almost never seems to manage to create this subtle balance between the importance of the individual parts, and the coherence of the environment as a whole. One or the other always dominates.”


Get every new post delivered to your Inbox.

Join 134 other followers