Audience Analysis, Critical for Instructional Outcomes

Written by Patti Shank PhD, CPT
It may not reflect all of the chat2lrn moderator opinions.

Audience analysis is part of the need analysis process during instructional design. The purpose of audience analysis is to help us understand who we are dealing with (including the organizational system) and how to serve them most effectively.

What Happens During Audience Analysis?

Some of the things considered during audience analysis:

  • Target audience: Who is the target (primary) audience and any secondary audiences? What are their expectations and needs? What problems are they experiencing? What is their level of experience? How much will they participate? How much time do they have? How will we respond, with instructional and non-instructional interventions?
  • Environment analysis: The entire environment people operate in. Leadership, learning, performance, business, competitive, work, tools, the entire system…
  • Instructional analysis: What tasks are needed to learn? Do people all know the same thing? How quickly does the information change? Is this declarative or procedural information? Is this information that people need to memorize?
  • Technical analysis: Does this involve technology? Hardware/software?  Will it be changing? How does it tie into company infrastructure? Who will deal with the hardware and software? Does the audience have the ability and capacity to deal with the technology and keep up with it? Who will build and maintain it?

Why Should We Perform Audience Analysis?

In “The Science of Training and Development in Organizations: What Matters in Practice,” Eduardo Salas and his fellow authors proclaim that “decisions about what to train, how to train, and how to implement and evaluate training should be informed by the best information science has to offer.”

I write about the critical nature of needs analysis for good training outcomes, according to Salas and fellow authors research in my ATD Science of Learning Blog article, Science of Learning 101: The Latest Research on Needs Analysis and Learning Climate (https://www.td.org/Publications/Blogs/Science-of-Learning-Blog/2015/07/Science-of-Learning-101-the-Latest-Research-on-Needs-Analysis-and-Learning-Climate).

Below is Table 2 from the Salas paper (http://psi.sagepub.com/content/13/2/74.full.pdf+html?ijkey=g8tvuLmoeZfN2&keytype=ref&siteid=sppsi), which shows that needs analysis is the key factor for maximizing training outcomes before training.

Table 2

 

 

Below, Table 3 of the paper, the needs analysis items that are most critical are clarified.

Table 3

 

The very first item, Conduct training needs analysis, includes (emphasis is mine):

Determine what needs to be trained, who needs to be trained, and what type of organizational system you are dealing with.

Is This a Training Problem?

by Patti Shank, PhD

I was very lucky when I was a young training manager and had the opportunity to learn with Geary Rummler (http://www.performancedesignlab.com/geary-rummler-founder). I truly believe that this training greatly helped my performance over the lifetime of my career. It provided a certain way of doing my work. The resource I will share with you will provide a brief synopsis of some of the thinking involved that I hope will intrigue you.

Why Care About This?

Training is an expensive intervention. We only need to provide training for one reason: People need skills they don’t have (or need to upgrade or re-establish their skills) and it makes sense to provide it in a formalized way.

When there are problems, such as people unable to do their jobs because of inadequate tools or not enough feedback about whether they are performing as needed (no performance standards), those problems must be fixed and training won’t solve the problem.

Example: A manager asks for team training for her staff because they are don’t work well together. In reality, she causes problems among them by how she treats them. She favors some over others. She provides more work and overtime to people she doesn’t like as much. Training might help this but she is the one that needs it. And before that, she needs coaching about the problems she is causing so the training might be valuable to her.

When we get requests (or demands) for training and we don’t determine if training has a good chance of solving the problem (or being part of the solution), we are creating a problem, not solving it.  Why?

We are using resources that could be better put elsewhere.

We are removing people’s time (when they are stuck in training) that they could be using towards better purposes. They could be using that time to get work done.
The problem doesn’t get fixed. (Think of all the resources used to not solve the problem!)
We look foolish and are unprofessional, and frankly, this happens too often. Who would hire a carpenter who couldn’t measure or build the right solution?

How Training Doesn’t Work

Example: When someone asks for customer service training but they have insufficient tools to answer customer questions or their process requires multiple workarounds, adding customer service training is a misplaced and expensive intervention.  They may need some training (or not) but they DO need better tools and an improved process so customers aren’t angry about being put on indefinite hold or sent to the wrong department.

Carl Binder’s discussion of Gilbert’s Six Boxes is a great introduction to thinking about what we need to do to have the type of performance organizations need and what influences these performance outcomes in the workplace. Read it and think about what part each part plays in your work. If you don’t think it fits in L&D’s world, we’ll have to disagree.

The Six Boxes: http://www.binder-riha.com/sixboxes.pdf

How Do We Do It? Crafting Decision-Making Practice in eLearning

Today’s post comes to us from #chat2lrn crew member, Meg Bertapelle. Meg is a Senior Instructional Designer for Clinical & Product Education at Intuitive Surgical. You can find her on twitter at @megbertapelle.

Decision-makingI think most of us would be happy to build more scenarios and practice decision-making activities into our elearning projects (time permitting) – if we knew up-front how to plan and execute them. Sometimes the hardest part is knowing where to start.

This week, I’ve asked a couple of our #chat2lrn community members to share their experiences crafting decision-making practice elearning activities. Fiona Quigley (@FionaQuigs), one of our #chat2lrn crew,  is Head of Learning Innovation at Logicearth, an Irish learning services company with a global client base. They specialise in the production of modern multi-device elearning content, learning technologies and training support services. Laura Payette (@ljwp) now works at Nielsen, but is coming off a three-year stint designing and developing elearning and corresponding product/marketing communication for KPA, a dealer services and internet marketing provider for over 5,000 automotive, truck, and equipment dealerships and service companies across the US. Her DOT Hazardous Materials course won the National Excellence in Training Award from the Automotive Training Managers Council in 2013.

We’ll get some great information from them in the form of an interview, then we can all discuss the pros & cons, and crowd-source some suggestions during the chat on Aug. 7th. I hope this will help us all take the initiative to help our audiences start applying their new knowledge & skills right away.


Q1: What kinds of decision-making practice activities have you been able to incorporate in your elearning projects?

Fiona: I’ve designed a lot of content for health and social care professionals, especially in the area of communication skills and policy compliance. I have designed scenarios that present a typical patient interaction and then ask the question – what would you do or say next? That would be what I call a level 1 decision making scenario.

Higher level scenarios are more immersive and instead of leading a leaner down a pre-planned path, they include random events that differ each time you ‘run’ the scenario. I’ve used these higher level scenarios with nurses and pharmacists in areas such as medication management – reducing errors and also for dealing with patient complaints.

Laura: I spent three years building elearning for the automotive industry, particularly in the areas of environmental safety and compliance. There was a lot of regulatory information that had to be included and I was constantly challenged to find ways to make it relevant and interactive. One of the ways I did that was to inject scenarios into the training. Keep in mind that many of them were on the smaller side. In other words, I didn’t build a course around one big scenario with a million branching options (although that would’ve been so cool!). My content simply didn’t lend itself to that. Instead, I used smaller scenarios and sprinkled them in where they had the most impact.

Q2: What kind of planning steps did you take before beginning to write the activities?

Fiona: You must talk to real people who do the jobs. Observing people making the real decisions is the gold standard – but it is often difficult to get the opportunity to do this. You need to be careful who you chose as the Subject Matter Expert is. Often SMEs are senior ‘expert’ people who are very far removed from day-to-day practice. To help people practice real decisions you must talk to the people who make the everyday decisions.  I also like to structure conversations with SMEs into what I call a ‘DIF’ analysis:

  1. Difficult – what if anything, do you find difficult about this decision
  2. Important – what is most important about getting this decision right/wrong?
  3. Frequent – what frequently comes up, e.g. common myths/misunderstandings, good practice?

Often competent practitioners won’t be aware of how they make good decisions. They are unconsciously competent; so it is the job of the ID to turn this tacit knowledge into explicit learning. Once that learning has been made explicit, you can more easily share that with others.

There is also a difference between formal and informal practice. There may be formal rules in place about how someone does their job – but many competent practitioners create shortcuts as they gain experience. Being able to identify these ‘tips and tricks’ is very useful learning in itself.

Finally, I would also advise talking to people at various levels of experience. For example, talking to a novice in the area will help you see the challenges first hand, rather than relying on the recall of someone more senior who may gloss over these challenges.

Laura: Research! Obviously, reviewing content from SMEs and talking to SMEs is critical but, like Fiona said, talking to people on the frontlines — or who at least aren’t far removed from the frontlines — really helps build context for understanding the challenges that employees face in doing their jobs. Sometimes that access can be hard to get; it was for me. If that’s the case, use everything you can to tease it out. Think of yourself like an investigative reporter. In my case, I had access to a robust database with thousands upon thousands of real-life examples that had been logged. I also had access to people who could elaborate on those examples to help fill in the gaps. I relied heavily on them and went back many times to ask additional questions.

Q3: How did you determine appropriate activities that would simulate the real-life application of your learning objectives?

Fiona: Again – much like the answer to Q2, observe decisions being made, find out how people actually make the decisions and base the activities on what actually happens in the workplace – not what SHOULD happen. Too often in elearning, we are forced to idealise and formalise the learning process, which then becomes so far removed from reality that it loses credibility with the target audience! You often see this in elearning content where the scenarios are so easy that you don’t actually need to complete the course to be good at them.

For example, when we designed a Medications Management programme, quite a few of our nurses said that one of the most difficult challenges they had was doing the ward round, handing out medication and being interrupted by patients or family members. They said they needed to concentrate and focus on making sure they gave out the correct medication – often a complex range of drugs for patients with very different medical needs. Another source of concern was worrying about patients who found medications hard to swallow and not having enough time to spend with them to help them. Together, we came up with guidelines about how to resolve these challenges and built in a scenario challenge around this.

Laura: The scenarios that I wrote were usually an outgrowth of the content development process. In other words, I didn’t approach a course with a specific scenario in mind. I generated them organically as I pulled the content together. It becomes apparent in talking to SMEs and frontline employees and in reviewing existing content where the gaps in understanding and practice are. Those gaps were usually the places I chose to insert scenarios because they illustrated the performance issue and allowed employees to think through things by answering the questions. In some cases with my content, there were right and wrong answers (remember, a lot of it was compliance based), but there were also usually shades of gray — and it was in those areas that I was able to challenge employees through scenarios to think about their actions and the ramifications of them.

Q4: How did you evaluate the effectiveness of your activities?

Fiona: I normally ‘dry run’ the decision plan with a selection of the target audience in a focus group setting. It is important to have a range of different people with different levels of experience. Role-playing the scenario, trying it on for size, works well to see if it is a realistic enough representation of the actual day-to-day-job. I normally use simple post-it notes to visualise the decision and focus on:

  1. Decisions – what is the actual decision to be made?
  2. Knowledge/Skill – what knowledge or skills do you need to make the decision?
  3. Actions – what specific actions do learners take to make the decision?
  4. Consequences – what are the results of each action, for both good and poor decisions?

Laura: I ran the activities by stakeholders and SMEs, as well as a core group of what I’ll call advisors for lack of a better word. (They were internal employees who interfaced directly with the external employees I built training for.) If they responded by effectively saying, “Oh, that really made me think about things differently,” or, “That really caught my attention,” then I knew I had hit the mark. If they didn’t, or if they were confused by what to do or how to respond, then I knew the scenarios needed more work. I know that’s vague, but there’s really no set recipe for scenario building; it’s very context specific. I also evaluated the activities by looking at actual evaluation responses from employees who took the course once it was deployed.

Q5: What made certain activities more effective/impactful than others?

Fiona: The more realistic the decision and scenario – the closer it is to the learner’s actual normal workplace activities, the better. Not only does the decision need to be realistic, but so does the consequence. We don’t want to use phrases like “Well done, that is correct” – rather, we need to show actually what happens in the workplace.

We have a challenge in elearning in that we usually have to design for a very generic audience. That means we lose the nuance and subtlety that actually drives high-performance. If you look at what drives and helps people to perform at a high level, it is mainly about understanding the subtlety of communication that goes on around you. It is also about reacting to unexpected happenings – like covering for a co-worker or working when you are understaffed.  We need to make sure we build in this nuance and realism.  To do this well, we perhaps need to have different types of scenarios to suit different types of people in our target audience. As learning designers we just can’t go on accepting a once-size fits all approach to our learners.

Also – a by-product to this analysis is that you need to be open to the fact that not all challenges that you uncover will be solved by training. For example, for our nurses, we identified that adding a simple “Drugs round in progress” notice to the drugs trolley, helped to reduce the interruptions staff faced. Identifying these possible environment or process problems is a great benefit of doing good decision making analysis. If you explain this to your client upfront, it can also be a great motivator for them to really engage with you.

Laura: Fiona makes some very good points here. I think including real consequences to real situations, and writing them in the parlance your target audience speaks, is key. If you fabricate your scenarios they won’t be authentic and people will dismiss them. They also have to be contextually bound. In other words, you may see a great idea for a scenario somewhere and think, “I’ll put that in my course!” But if you don’t mold it for your audience/content and their specific performance needs, it won’t be a great scenario for what you’re building. I think sometimes the scenarios that are most impactful are those that address gray areas — the places where employees are a little uncomfortable or uncertain — and the places where the biggest performance gaps are.

Q6: Please share your top tips/tricks for crafting decision-making practice activities.

Fiona: I think I have covered most of these in answering the questions above, but to summarise:

  1. Talk to real learners of different levels of experience.
  2. Be aware of the formal way of doing something versus the informal way.
  3. Help your SMEs make their decision making practice more explicit by asking good questions.
  4. Have a range of scenarios to suit different types of people in your target audience.
  5. Dry run your scenario plan with representatives of the target audience and adjust accordingly.
  6. Find out why people are making common mistakes e.g. is it a process or environment problem rather than a training problem?

Laura: Fiona’s tips are great. The only thing I’d add is be sure to craft your scenarios in the language your target audience speaks so they sound authentic.

Thank you to Fiona and Laura for sharing their insights. What about you? If you have some experiences and insights to share, or just want to hear what others may have to say, please join us Thursday, August 7th for #chat2lrn at 8am PDT, 11am EDT, 4pm BST.

New Year’s Goals, Plans & Wishes

MP900309665Happy New Year!

Welcome back from the holiday craziness.  We hope you had wonderful holiday and new year celebrations and are all settling back into the groove of work.

Before you get too settled though, we’d like to stop and think about what we’d like to bring to & get out of 2013.

What are your goals for the year? What are your plans? Have you even thought about it, or are you just trying to catch up and taking it 1 day at a time?

Personally, I find it difficult to take the time out of the daily grind to think of the bigger picture and what I want to accomplish.  I do know that I need to carve out the time to get and keep myself organized.  When I get busy, I start letting that slip, and then I get more stressed because there’s so much to do AND I feel scattered due to the lack of organization – so that’s high on my list.  I’d also like to protect some time to reflect on things I’ve learned, and do some exploration and experimentation with those new ideas in my head.

Many of our colleagues have already thought of several things to think about, maybe they’ll be inspiring to you:

Let’s take some time together this Thursday to inspire each other!  Please join us Thursday, Jan 10th at 8am PST/11am EST/4pm GMT to share our goals, plans, and ideas about the fresh new year before us.  Hope to see you there!

~Meg

Meg Bertapelle

Are you supporting performance?

All organisations are measured by their performance.  Measures of success vary, but all successfully performing organisations have business strategies in place that allow them to survive and sometimes even grow during tough economic conditions.  As employees and businesses begin to leave the comfort of “things known” and venture or are forced out into the untrodden lands of uncertainty and unfamiliarity, their need for support increases and  our ability to deliver appropriate support is increasingly challenged.

In our now constantly and ever more rapidly changing world, the recognition that learning is ubiquitous is now widely accepted, as is the recognition that it cannot be managed. So what is the place of a learning expert or a learning function in this new world? What does support mean and how can it be provided?

Learning requires to be inextricably embedded in the work stream. At its simplest people need to have the support tools or “sidekicks” as they are described by Allison Rossett  and Bob Mosher to help them perform tasks. Such tools need to be incredibly practical and accessible. It is often forgotten that learners are in complex situations back on the job and regardless of the quality of the training, application is always more difficult when in the work stream and away from the protected environment of the learning intervention.

Supporting peoples performance in the workplace can and should happen using the whole span of technology and learning theory that is now so easily available to us.  Whether it is a simple checklist of codes at the self-checkout in a fruit and veg shop or a pilot’s complex pre-flight procedure, or maybe a simulation for a surgeon to practice a rarely used operation before treating a patient, the tool needs to be directly applicable and be focused on successful performance of the task and the achievement of desired results. Access to a co-worker skilled and experienced in the task or to an expert coach is another form of performance support.

At an organisation level, recognition of the predominance of informal and social learning needs to be given prominence and supported accordingly. All of us who work in learning need to concentrate fully on understanding the business sufficiently so that we can apply our knowledge to performance and improvement of results, whether at an individual, team or organisation level.

It goes without saying that human capital is an organisation’s greatest and most fundamental resource, therefore for an organisation to perform at its best, its members of staff need to be performing to the best of their ability.

The more we can support people out of an understanding of their own and their organisation’s needs, the more likely it is that a beginner will become an expert, a follower will become a leader, and stagnation will transform to innovation.

So how well are we doing? In 2010 Capita asked senior business leaders in the UKs largest firms how learning and development was contributing to the organisations ability to perform.   The outcome of the research is worrying with only 18% of leaders saying that L & D strategy aligns to the overall business strategy.  There may be many reasons for this, but we have to address them. A recent unpublished American survey revealed the horrifying statistic that only 15% of executives would recommend L&D to their colleagues as a resource in improving business performance. We are a very long way from delivering the added value to the people and organisations who employ us, both in their perception and in our aspiration. If we fail, we will become isolated and increasingly irrelevant overheads that organisations will no longer afford.  Succeed, and we become central to the future.

#chat2lrn on Thursday 16 February will seek to explore what performance support means in practice, and how as learning professionals we can position and skill ourselves to provide that support…..hope you can join the conversation!

To help you get ready for our next exciting session, have a look at the following posts………….

http://www.learningsolutionsmag.com/articles/668/the-other-side-of-learning-performance-is-everything

http://www.trainingindustry.com/blog/blog-entries/where-does-learning-end-and-performance-support-begin.aspx