The Meta-Learning Lab at eLearning Forum
by Jay Cross, Internet Time Group
The Meta-Learning Lab presented its core beliefs and findings at the March 2002 meeting of the eLearning Forum at the SRI campus in Menlo Park, California. This is a recap of the event.
|
Jay |
Bill Daul, the networker Jim Spohrer has described as “human glue,” encouraged
a group of East Bay denizens to meet one another for lunch about six monnths ago.
(Bill is the founder of Human Landscaping.)
Several of us were members of eLearning
Forum, and we think of ourselves as an eLearning Forum task force. Discovering
we each believed in the untapped potential of learning to learn, we decided to
meet to talk things out in Tilden Park. We
continued meeting at one another’s houses and began meeting with learning visionaries
at ethnic restaurants in Berkeley.
Sensing we were onto something that might change the world, we developed an infrastructure early on. Groove was a bust – a couple of us don’t like using the phone that much, so voice over IP was not really a plus. Our threaded discussions on EZ-Board were too easy to overlook; comments were more episodic than interactive. Blogs were too esoteric for the group to buy into.
|
|
Bill Daul |
Group session at Clark's house. |
QuickTopic, the free bulletin board which lists postings chronologically, proved to be a winner. Our private discussions are now up to 850 posts. We set up a knowledge base which is a useful reference but participation there is spotty. We also put together a public website at www.meta-learninglab.com.
What held this group together? Aside from the fact that we like one another, unlocking the possibilities of learning to learn excited us. Our discussions linked us to one another's favorite articles and stories.
Largely self-taught, I was soon devouring great source material on learning. Claudia L'Amoreaux inspired me to read Deschooling Society (which I absolutely loved) and to re-read John Bransford's How People Learn. Clark Quinn led me to Don Norman's Things That Make Us Smart, cognitive apprenticeship, and The Nuruburg Funnel. Claudia Welss became my touchstone for finding deeper meaning, setting the right tone for our inquiry, and listening to what intuition is telling us. I learned from reading Reigeluth, Maturana, Visser, Abbott, Senge, Spohrer, Banathy, Lauren Resnick, and a host of others.
Each of us learned from one another and also linked from one topic to the next individually, guided by the joy of discovery.
So what is this meta-learning? Meta-learning focuses on improving the process of learning, including how people learn, barriers to learning, and improving the learning of both individuals and organizations.
Meta-learning analysis must be multidisciplinary, branching to other disciplines to escape the deep ruts worn by schools and higher education. Drawing from direct observation and down-to-earth metrics, meta-learning researchers borrow approaches from anthropology, psychology, economics, learning theory, cognitive science, sociology, political science, operations research, information architecture, and software design in their search for improvement.
We’re on a crusade to rip the blinders off!
I just returned from the lovely colonial capital of Antigua, Guatemala. Facing
the main square is an enormous building, once the administrative headquarters
for all of Spanish Central America. A colonnaded balcony looks out over the plaza.
Permit me to use the balcony as a metaphor. Balconies are great vantage points. From the balcony, we can peer down into the main square and observe people learning.
Elders talk with youth. Children learn through play. People share knowledge. These particular learners happen to be pure Mayan but that’s beside the point. Learning is universal, and the bulk of it is informal and social.
The balcony’s owner has kindly set out an array of binoculars, telescopes, opera
glasses, telescopes, and spyglasses. These are no ordinary lenses. Each enables
us to observe through the eyes of a particular profession. You can look through
them to get an anthropologist’s view of things, or a cognitive scientist’s, or
a psychologist’s, or a business manager’s. We see the patterns of what’s going
on in learning, formal and informal. All of which brings us to today’s topic:
meta-learning.
We’re not going to deal with meta-learning in the abstract. Our interest is in applying the meta-learning perspective to business problems.
Out topic is Applied Meta-Learning.
As a laboratory, we want to conduct experiments on meta-learning as well as disseminate results of research. Or so we said....
At the event, we took the group from their familiar configuration sitting around the screen,
to one of the grouped tables in the back.
There, they were told:
Together with those at your table, design the major components of a new employee induction program
Once they got organized and going on the task, we broke into their process and instructed them to go to a different table, with the most new people.
There, they received the following instructions:
Now take turns describing how you organized yourselves to design the induction. The process, not the product
We let them take some time to discuss how their groups had talked. Then we returned them to the original room configuration.
We asked for insights:
One report was that the individuals at the table competed to get their ideas out. Another reported cooperative efforts to contribute. Interestingly, the distinction was observed to be slight.
Another report was that they did not find it difficult to talk in terms of process instead of product. Hey, real data!
Of course, what we wanted was to help the participants actually experience process reflection. Focusing on using the process perspective to improve the process is applied meta-process. Of course, our interest is most specifically on learning as a process to be investigated and improved. Voilá, Applied Meta-Learning!
What I’m about to say is controversial. Certainly, not everyone buys it. The topic is Mindful Learning. The thesis is from Harvard’s Ellen Langer. An experiment is probably the best way to describe what this is.
Two groups of college students, one on the east coast and one on the west,
were given the exact-same paper and were instructed to read the paper attentively
because they’d be tested on the topic.
There was one difference. One group was told the information might not be true.
That group, the one who were told the information might not be right, scored significantly higher than the other group. Why? Because uncertainty engages the mind.
Instead of prefacing eLearning programs with authoritative plugs, designers would do better to begin with “This might be wrong,” and perhaps a contest for spotting typos.
People learn from their mistakes. And their strengths. All it takes is reflection.
Remember the Chinese proverb, “Give a man a fish and he will be fed for the day. Teach a man to fish and he will never be hungry again.”
The Meta-Learning Lab doesn’t stop there. We want to change the process. We say, “Teach the man to be a better fisherman, and his family will be always have plenty to eat.”
Here’s an excerpt from my personal weblog:
A key aspect of improving how one learns is to make the learning process explicit. When you make it visible, you can exercise the skill, raise the bar, and improve performance. (This could also be thought of as "getting over learned helplessness.")
By paying attention to a suggestion made by a friend, I realized that for me, sleep produces great ideas. Now I awake expecting them. Before my first cup of coffee, before bringing in the New York Times, and before checking my messages, I let the new ideas pop out. I record them in a blog, sometimes private, other times public.
This morning brought a few ah-ha's for the upcoming presentation to eLearning Forum. One of them was to show this very log as a way I make my own learning processing more explicitly. Until working with the Meta-Learning Lab, this blog was titled "Just Jay."
When I returned from vacation , I received an email from a friend Marcia Conner, asking about the effectiveness of eLearning. Here’s my reply, which I posted in my weblog on Research on Learning and Performance:
i got in from guatemala last night and will use that as an excuse for a wimpy answer to your question. (photos and commentary from a great vacation are up at jaycross.com)
the older i get, the more i trust my gut and the less i believe statistics.
my gut tells me that elearning sometimes works well. for the successes, i point to my son and his peers learning everything from homework assignments to network administration via the web. that's also where he learned a lot more than his dad ever did about meteorology, PERL, San Francisco politics, environmental action groups, obscure singers, and more. in my own case, i've learned more professionally from Amazon and Google in the last seven years than from a similar span at Princeton & Harvard b-school. at smartforce we accelerated sales development by the better part of a year; it wasn't perfect -- it wasn't even pretty, but it was at least $10 million more cost-effective than the cram sessions it replaced. in the informal arena, a lot of what my generation learned on the playground is now learned via instant messenger. so my gut says elearning works well.
of course, i've seen cases where massive elearning projects fell flat. my comparison of elearning to napoleon's march to moscow came up with only 10% of corporate elearners achieving their objective. i've attempted courses on communications and negotiations and customer service that were so awful you'd have to be brain-damaged to sit through more than five minutes' worth. i'm not convinced that a two-hour CD on sexual harrassment is going to tame the hormonal urges of preditory beasts. ASTD and Elliott found that a third of the recruits don't even show up for *mandatory* elearning. someone was telling me about the courseware peddled by the american bankers association. several large banks bought licenses, citi among them. how many participated? not one person. over the course of a year. a researcher at SRI described a $2 million corporate university program that attracted only two participants. both of them dropped out. so my gut says elearning can fail miserably.
sometimes it works; sometimes it doesn't. just like computers. geez. makes me ask "how effective is high school?" i graduated with high honors but damned if i can remember a single formula from trig. speaking french with the owner of my hotel last week, you'd never guess that i finished French IV. looking at the general populace, half of all u.s. high schools grads cannot locate france on a map. more than half of the males attending california state universities and colleges do not possess sufficient skill to read the textbooks in their backpacks. if i'd skipped high school entirely, i wouldn't be able to quote t.s. eliot or tell you the year the magna carta was signed or name the capitals of every country in the world, but i don't know that it would have hurt my career. i learned more about mayans and guatemala in the last ten days than i would have from a three-unit college course. there's a reason that grades in college do not correlate to income, happiness, professional accomplishment, or anything but getting into schools that look at GPAs as an admission requirement. don't get me started on latin.
so i agree with all of you. it's a stupid question. i don't question the effectiveness of my hammer. if the nails go in, it works. if it's the best tool around, i'm not about to try pounding those nails with my shoe or a screwdriver.
god i'm tired.
As you can see, I’m not a big fan of organized schools. I emailed my words to the group (“Reply All”), and the next day received a very reasoned response from Marc Rosenberg:
...most of the examples Jay points to of elearning working are NOT COURSES. If fact, the learning comes from somewhat intelligent (on the human side) browsing for something that actually means something to the browser. I understand where courses are the best solution, but there is no way courses are the ONLY solution. So why is everyone peddling courseware? Habit, I guess. Perhaps the need to have something you can "evaluate" with multiple choice questions. Or the need to have something people can "enroll" in.
Marc’s insight sparked further thoughts, which I once again recorded in my blog:
bingo! the course is not the appropriate shell for most learning experiences. we all know the story: the fifty-minute hour and the two-day workshop were created for the convenience of the institution, not the learner. the course is a triumph of standardization and it is so ingrained in our thinking that we still buy & sell seat-time rather than performance improvement. it's the industrial model, which puts a higher value on efficiency than on effectiveness. you can have learning any color you want as long as it's black.
in an ideal world, the less time in training to accomplish the objective, the better. optimizing time invested requires a new way of thinking about learning. essentially, learning is a flow process, not an assembly line.
think of learning as a river. i can use calculus to describe the rate of flow at any point in the river by mentally divvying up the river into lots of little pieces. this provides useful descriptive information, but back in the real world a time-slice of a river is impossible to deal with. outside of the context of the current, the little chunks simply don't exist.
we need a better framework for thinking about learning.
Imagine that you are chief learning officer for a Silicon Valley high-tech company.
You read about a new CD-based course that’s just come out: Mavis Beacon Teaches Reading. It takes four hours to complete. It costs $39 retail.
This is not a remedial course, Bonehead English in a Box. Exhaustive research has shown that highly literate knowledge workers improve their reading and comprehension an average of twenty percent.
Would you, as chief learning officer, put Mavis Beacon Teaches Reading into your core eLearning package?
Of forty people in the room, not a one was going to endanger his or her career by implementing Mavis. Probably a smart move. Others would question the value of a program in reading. What about leadership training instead?
Consider the economics. How much time does a knowledge work dedicate to reading reports, webpages, newspapers, email, journals, and memos? It’s at least two hours a day. That’s about 500 reading hours a year. Or 5,000 hours a decade.
Assuming conservatively an annual salary of $68,000. Add taxes, insurance, fringe benefits, and a little overhead. This individual costs us $100,000 a year.
A twenty-percent improvement is a savings of 1,000 hours. That’s a savings of $4,000. Not a bad return on $39.
But the numbers are flawed. Knowledge workers are the source of innovation. They are expected to do more than just earn their keep. So we should really be assessing the increased value associated with 1,000 newfound hours rather than the cost of those hours.
No
matter. No matter how you slice it, improving reading skills is a spectacularly
sound investment. So why didn’t our Chief Learning Officers choose to implement
Mavis Beacon? I’ll suggest that it’s time myopia.
In the homo sapiens’ long evolution from dwelling in caves to living in condos, the last ten thousand years are but a blink of the eye. Human brains are better suited for avoiding tigers right now than for thinking ahead. Our mental circuits are wired to emphasize the short term.
If you’re only thinking ahead a few weeks, taking the reading course is not a sound decision. There’s insufficient time to reward the investment. By looking at things from different perspectives, meta-learning attempts to cut through the bias of snap judgments and fuzzy thinking.
One perspective we can (and I do) apply is an applied cognitive (or cognitive design) lens. In this lens, we take what we know about how people think and learn, and use that as the basis for understanding and improving systems.
 Clark |
My own meta-learning moment came when I realized that the note-taking behavior I had used in college, where I reread them (typically the night before the exam), had carried over post college, except I was no longer re-reading the notes. So, why take notes?
I asked what meta-learning moments the audience had. One response reflected that when trying to use a fax machine, only one of the group actually went to the manual, while the others attempted to reason or trial and error their way through.
For myself, I still take notes, I still don¹t re-read them, and I now think it¹s a good idea. The applied cognitive explanation is that I deliberately try to rephrase the statements in my own words, and often I draw diagrams. This is extra processing that we know leads to better retention. I am trying to get back to drawing mind maps.
In this case, to have the background to consider our own learning, we need a representation for learning (why is explained in the diagram itself!).
This one, based upon Cognitive Apprenticeship, includes elements we know facilitate learning:
This model accounts for formal learning, but to account for how people really learn in the world, we need to go further.
This framework (based upon activity theory and back to hermeneutic philosophy) captures how people act in the world:
Individuals normally are engaged in well-practiced activity, where their existing knowledge allows them to continue.
Occasionally (and increasingly, in this era of change), individuals get into situations where they have information needs (known as 'breakdowns'). If they can find the answer, they may return back to activity, without actually learning anything.
Sometimes they can¹t find the answer, and have to actively problem-solve.
If they do and reflect on that process, they may actually learn something. However, learning is an extra step that isn¹t necessarily performed.
Personal story: I regularly rediscover how to fax from my computer because the trial and error process is easier than attempting to discover and retain a useful conceptual model for my fax software. So, I¹m NOT learning, though I regularly problem-solve.
With these frameworks, we can look at learning situations:
In one case, we might have a learner with an information need, who¹s gone to a search engine and typed in a query, and is perusing the results. This may or not be a good strategy, depending on whether the answer is likely to be on the web, whether the choice of engine is appropriate, whether the individual knows how to write a good query statement, and is good at processing the results.
In another case, we have someone who¹s gone to ask someone in the next cubicle for help. Again, this may or may not be a good strategy, depending on whether the person asked for help knows the answer, is good at explaining it, has a vested interest in being correct, and whether the recipient is good at processing spoken responses.
By examining our learning, we can then apply ourselves to optimize our learning. Integrating the frameworks from before to understand self-learning:
This framework suggests that we need to have a record of activity, which we can compare to either a modeled example, or generate one from a conceptual representation. Ideally we have both.
In conjunction with the framework, we need a process:
We need to know our areas for improvement, prioritize them, and choose which one we¹re currently working on. We need to plan by finding an example or representation, as well as an opportunity to practice. We can put in place support for remembering to focus on a skill during the upcoming event. We then need to execute our task, both accomplishing the goals of the task as well as remember the item we want to improve. Finally, we should remember to reflect upon our performance, either from a record or from memory, and evaluate whether we¹ve improved or need more work.
To make this concrete, imagine we want to work on our coaching, and find a framework that indicates an important component is listening, which we recognize we could stand to improve. So we identify our next coaching opportunity, and put in a reminder to listen before hand. We go through the coaching activity, remembering to pay extra attention to listening. We¹ve also put in a reminder to review, so we remember to reflect afterwards. We may not be able to record the session, so we¹ll have to rely on memory to review our performance and see whether we accomplished the goal of ensuring we listen to the employee.
This framework for self-improvement on a process can be applied to our own learning through technology:
Here I¹ve made two jumps, applying the self-learning framework to technology support and to self-learning. We see we need to present learning concept representations and modeled examples, as well as capturing learning activity. And the system can provide support for remembering to reflect on learning.
The point is that just as we use formal learning models to design our formal instruction, we can and should apply broader frameworks to help us improve our working activities and learning. This meta-learning perspective promises to optimize elearning investments, and to bridge the gap between elearning, performance support, and knowledge management applications.
The Meta-Learning Lab welcomes your participation. Here’s how we are structured.
The core group maintains the identity and intellectual capital of the Lab.
Current members are Jay Cross, Clark Quinn, Bill Daul, Claudia L’Amoreaux, and
Claudia Welss.
The community consists of advisors, colleagues, and co-conspirators. Sign up for our mail list at www.meta-learninglab.com.
Ad-hoc project teams are where the rubber meets the road. We learn by doing. We field-test everything we do. Currently we are seeking developmental partners. Our target customer is an organization that is unafraid of innovation, willing to experiment, and is sufficiently large that meta-learning can yield at least a million dollars in benefits. Typical assignments include:
For more information, visit us online at www.meta-learninglab.com.
The core group: Claudia Welss, Clark Quinn, Jay Cross, Claudia
L'Amoreaux
