Dec 18 2011

Thankful

Have you ever been at a social engagement and found yourself chatting with a new parent? It seems that no matter the topic, all that they can manage to talk about is their child:

“So they changed our garbage service”
“My child got into the garbage the other day. It was so gross and funny.”
 
“I heard the NBA strike is over”
“My son is in the 95th percentile of length. Even though he is 18 months old, I have a strong suspicion that he will be a baller.”
 
“I went to have my oil changed…”
“Baxter drools when he smiles for too long.”

Two years after starting Word Lions with Philip, I now empathize with that parent. Talking to me about anything other than content creation, instructional design, or running a small business is akin to trying to have a conversation with that new parent. Their baby always makes its way into the conversation—sometimes without a thread of relevance.

I have become a broken record, and I love it. I am so deeply thankful for my life and my work:

  • I am thankful that my work is to provide valuable, meaningful content for people who need it.
  • I am thankful that I can write for a living.
  • I am forever thankful that I can partner with the inestimable Philip Morgan.
  • I am thankful that together we are growing a company designed around our authentic selves. That when I am working I am able to be myself.
  • I am thankful that we prove every day work can integrate with life.

Growing Word Lions is not easy, but it is constantly interesting and meaningful.

When you talk to a new parent, and ask them how they are doing, they seem to always simply say

“I am tired. All the time.”

It seems unbearably difficult. Now, raising a business, I think I understand why new parents are smiling when they say that.


Oct 17 2011

Shiny Disks vs. HTTP: electronic training methods compared

Word Lions recently helped a customer author a CD-ROM for some training content that our customer created. During the process, we were reminded that:

  1. We’ve created a lot of training content that was ultimately distributed either online–via a Learning Management System (LMS) or web page–and via optical media like CD-ROM or DVD.
  2. We’ve had to learn a lot about how to do optical media content distribution through difficult on-the-job lessons.
So, we thought we’d put together some lessons learned and best practices to help other folks who might be facing the task of widely distributing training content.

laserdisc.jpg

Shiny Disks or Web Pages

You’ll want to decide whether you are going to distribute your content on physical media (shiny optical disks) or electronically (using a LMS or web page) early on in the content development process. Why? Because the Quality Assurance (QA) process is very different depending on which distribution method you go with, and getting the QA right will make or break the customer perception of your training content.

You decide whether to use CD-ROM or HTTP by thinking through the constraints of your training environment and other relevant considerations, summarized in the table below:

Constraint/consideration

Lean Towards

Why

No reliable internet or network access in the training environment.

CD-ROM/DVD

Trainees won’t have access to an LMS or web page if there’s no internet access in the training environment.

Training content will change often and students need immediate access to the updated content.

LMS/Web

An LMS or web page lets you deploy updated or changed training content with much greater speed, ease, and lower cost than a CD-ROM/DVD distribution.

Your content is already behind schedule and you have a hard date you’re trying to hit.

LMS/Web

If your project is already behind schedule, the extra time it takes to move ISO files back and forth, burn images, and do a good QA job on the resulting disks will further threaten your delivery date.

You need to control who has access to the training content.

LMS/Web

After a CD-ROM/DVD gets into the wild, you have no control over where it goes or who sees it. With an LMS or Web page, you can use DRM and other techniques to control who has access to the content, and you can revoke access if necessary.

You need to track who uses the training content.

LMS

Almost all LMS systems have some ability to track who uses the training content, and the outcome of their interaction with the content and any assessments it contains.

The audience for your content has a particular preference either for Web or optical media methods.

Depends

Do take into account your audiences’ preferences!  Some people want a physical disk they can hold in their hand, take home with them, and cuddle up next to at night.

Optical Media Best Practices

If optical media is the best distribution method for your training content, keep the following best practices in mind:

  • We recommend that customers plan for the same kind of QA procedure you would have if you were distributing software on optical media. In general, this means having at least:
    • One QA pass on the finished content before you create a master CD-ROM/DVD image.
    • One QA pass on the finished content after you use the content to create a master CD-ROM/DVD image.
    • Testing the content end-to-end on one or more computers configured exactly like those your customers will use.
    • Allow adequate time for CD or DVD duplocation and shipping.
    • Allow some time (and budget) for the unexpected!

LMS/HTTP Best Practices

if an LMS or web page is the best distribution method for your case, keep these best practices in mind:

  • While we don’t encourage skimping on QA if your are using an electronic distribution method, you certainly can fix issues that your QA process uncovers more easily than with optical media. For this reason, you may want to structure a beta release of your training content–via web or LMS–to a limited audience as a way of testing your content before it’s wider release.
  • While there are lots of nice LMS packages that you can self-host, give serious consideration to hosted solutions. Hosted LMS companies can save you a lot of setup, configuration, backup, and other operational headaches.
  • Even though an LMS makes lots of things easier, there will be support calls. Create good instructions on using the LMS for your customers, and a good FAQ, to reduce the number of support calls.

Oct 12 2011

Lead With Benefits and Value

Today I purchased a couple of things at some different stores. Twice I was asked whether I was “in our system.” To the surprise of the clerk, I answered, “what does it matter?”

In both cases, I ended up asking the clerks a simple question. “What is the benefit to me of being in your system?” In both cases, the clerks had difficulty articulating how I would benefit from taking additional time to complete the sale by giving them my name and address so I could be added to their customer database.

While I’m not surprised that neither clerk could articulate the value or benefit of being in their customer database, it did remind me that this sort of thing happens all the time in the technology industry.

Some companies assume that it is enough to do a good job of describing the features of a product. This is equivalent to the clerks who–in the form of a question–said, “One of the features of our store is a customer database. Would you like to use this feature? It will only cost you two minutes.”

This approach puts the burden of translating the feature into a benefit or value on the customer, who is often ill-prepared to make this translation. Why didn’t these clerks instead frame the feature in terms of its benefits? And not its benefit to the business; i.e. the ability to send me advertising, but in terms of its benefit to the customer. Something like, “If you’re in our system, we can process a merchandise return twice as fast. Are you in our system?”

Remember that while it’s good to describe features, it’s better to lead with the value or the benefit of those features. Save for the feature descriptions for after you’ve hooked a prospect with the value or benefits of your product or service.

If you want to discuss ideas for using content marketing to get the word out about your product, drop us a line at TheLions@wordlions.com.


Oct 3 2011

Things We Love: The good feeling of oneness with cup rubber

This is an update to one of our most popular posts: Things We Love: Clicky keyboards.

A few months ago, I ranted about my affection for mechanical keyswitch keyboards. As a writer, having an accurate, responsive keyboard is important, and I was able to meet that need with the relatively inexpensive keyboard Solidtek ASK-6600.

Well, the urge to keep exploring high-end keyboards struck again, and I got a Realforce 86U. This keyboard uses the highly-regarded Topre rubber dome keyswitch. Rubber dome keyswitches are used in some truly awful keyboards, but as it turns out, that does not mean that rubber dome keyswitches are themselves awful.

In fact, they can be a real delight. I love my Realforce 86U! It makes less noise than my Solidtek ASK-6600, it has an ergonomic design that works very well for fast, accurate touch typing, and a springy, precise feel to the keyswitches.

It’s a spendy little keyboard, but I can see myself being very happy with this keyboard for a long time!

By the way, “The good feeling of oneness with cup rubber” is the hilariously garbled english tag line printed on the Realforce product box.


Sep 28 2011

Improving Assessment Performance and Design: Part 6

Now that we’ve provided some pretty detailed guidance on how to exploit flawed assessments, we’d like to take that same knowledge and provide some guidance on designing better assessments; ones that don’t have easily-exploitable flaws that savvy test-takers can use to distort the assessment results.

Avoid Obvious Structural Flaws

If your assessment includes multiple-choice questions, you’ll want to avoid grammatical inconsistencies in how the choices are written. As you’ve seen from earlier installments of this series, savvy test takers can exploit patterns in the grammar of your questions to help identify wrong answers.

Best Practice: After writing the assessment, look at the grammar of multiple-choice answers. Verify that there are no grammar patterns that correspond with correct or incorrect answers.

Example: 

Question: What is the correct order of steps for starting your car’s engine?

A. Insert the key, secure your seat belt, verify that the driver’s window is cracked open, and then start the engine.

B. Insert the key, check the rear-view mirror, start the engine.

C. Secure your seat belt, check the rear-view mirror, insert the key, start the engine.

Note that answer A ends the comma clause with “and then,” while B and C use a different structure. This inconsistency in parallel structure could be enough to tip a savvy test taker that A is either the right answer or a poorly written distractor.

When writing questions, structure the questions as consistently as possible. By doing this, you focus on assessing knowledge or skill rather than forcing students to waste time compensating for variations in the question structure or wondering whether variations in question structure are meaningful or not.

Best Practice: After writing the assessment, read all the questions. Verify that they are reasonable consistent in their structure. Example structure: Describe a scenario, provide additional important details, then ask how the learner would solve a particular problem related to that situation.

Example: 

Question 1: You are in a dark alley. You hear footsteps behind you. What should you do?

Question 2: You are in a foreign country. A stranger asks you to deliver an unmarked package for them. What should you do?

Question 3: What should you do if you get lost in a strange city and someone asks you for money?

Question 3 uses a completely different structure than the other two. This can be confusing to the person taking the assessment.

In some contexts, patently wrong or silly answers can help identify random guessing attempts, but on assessments designed for an adult audience, silly answers have no place.

Best Practice: Avoid silliness when writing wrong answers. A wrong answer that is purposefully silly is insulting to your audience and reduces the quality of your assessment.

Design and Implement an Appropriate Mix of Conceptual Knowledge

It’s easy to create an assessment that targets knowledge of trivia. If it’s software knowledge you’re assessing, it’s very easy to determine whether the learner knows where to click to accomplish a desired outcome. If it’s customer service telephone skills you’re assessing, it’s very easy to determine whether the learner understands that it’s better to refer to a caller as Sir/Madam, their first name, or Dude.

What’s more difficult is to assess conceptual knowledge. But before you attempt to create an assessment that tests for conceptual understanding, make sure that’s audience-appropriate! In other words, you need to circle back around to your training objectives. You did design your training around defined objectives, didn’t you? Your assessment(s) should support the same objectives your training was also designed to support. In fact, this is what assessments do: verify the success if your training.

Best Practice: Map assessment design to training objectives. 

Avoid the Other Common Assessment Flaws: Excessive Trivia and Edge Case Interest

Again, assessing conceptual knowledge is more difficult than assessing understanding of trivia. This can result in assessments that have an excessive percentage of questions that are designed to test:

  • Trivia: obscure details of the subject you are assessing that have little or no relevance to your training objectives.
  • Excessive interest in edge cases: Unusual situations that pertain to the subject you are assessing. Example: You are an astronaut. How long can you survive the vacuum of space without a space suit?
Best Practice: Read through the finished assessment. Classify each question based on whether it is a skill, knowledge, detail, trivia, or edge case question. Obviously, this is a subjective call. Determine whether the breakdown of question type is appropriate to training objectives and audience. Redesign or rewrite questions as necessary.

Finally, Assess Your Assessment

Despite your best efforts to write a good assessment, you may end up with some questions that are clunkers, poorly written, or are otherwise invalid tests of knowledge or skill. The best way to deal with this is to beta test your assessment. Administer your assessment to a relatively small but representative group of people; perhaps 5% or so of the expected audience. Look for patterns in the results:

  • Outliers: questions that almost everybody or nobody gets correct. These are probably flawed in some way and need to be redesigned or rewritten.
  • Result distribution: What is the distribution of assessment outcomes? Are you trying to fit the outcome to a predetermined distribution, or meet some other criteria?

Conclusion

Good assessments are an important part of any training effort. By paying attention to consistent grammar, consistent question structure, and by mapping your assessment design to your learning objectives, you can create assessments that provide maximally useful feedback for you and your learners.