What is the best way to assess staff skills?

Recently I got an email from a librarian from a library I’d consulted for in the past on technology training.  Her question was one I hear a lot, actually: “We need to create an assessment of our staff members’ skills in different areas.  What is the best way to get this information about them?”

My answer is really simple.  Ask them.

To back up a small step, you do have three primary choices when doing a staff assessment of any skills.

  1. A test: staff are given some kind of computer or human graded “objective test” of the skills, usually timed
  2. A peer walk-through: staff member has to perform each skill and a co-worker (sometimes the person’s supervisor, a trainer, or expert in those skills), marks whether they know how to do it or not
  3. A self-assessment: staff are given a list of skills and asked to report whether they know each one or not

If you want to make your staff really, really angry with you and waste a lot of time and money, by all means go with #1.  People don’t appreciate being tested and I promise you that the staff en masse is more likely to react negatively to any further training provided if you go that route.

I’ll admit that #2 can certainly work and be accurate, but puts people in a position of feeling judged by someone they work with.  This can be awkward for both parties.  You can certainly make the argument that an employee should sometimes feel judged by a supervisor, but just as with the first option this can create opposition to any steps toward training or skill development that come after the assessment.

So we’re left with #3: the self-assessment.  Ask staff if they know how to do what you want them to do.  Give them three choices: yes, no, or maybe.  And for trainers’ purposes, a “maybe” counts the same as a “no” because it likely means the person still needs training.  Answering “maybe” is just a whole lot less threatening than answering “no” for some people.  And tell them up front that they’re not expected right now to have every single one of these skills, and that there are no penalties for their answers.  What matters is that we get an accurate baseline for the system so we can provide the right types and numbers of training for the right people to make sure that everyone has the chance to learn and improve the skills we use every day in our jobs.

I always like to tell people upfront as well that their results will be shared with their direct supervisors.  Supervisors should know where their employee’s skills are, but this can help them get a more concrete view.  Also, this supervisor layer adds a filter to catch the inevitable “exaggerators” — people who either hate training and the skills at hand and so lie to avoid it or people who are still afraid, no matter how much you reassure them, so they bluff that they know things they do not.  Supervisors will know if a person has just outright not been honest on the assessment — and they can act as intermediaries so that the trainer isn’t the one approaching the person saying “uhh, you don’t actually know this.”  Giving those few exaggerators a chance to re-take the assessment once called on their bluff is helpful to everyone involved.

All in all, the most important thing for a trainer is getting accurate data about who needs what training and to be left with a group of people willing to receive that training.  In my experience, the self-assessment is the only way to go.  But I realize my experiences are limited, and so I turn to you!  I’m curious to hear about other people’s experiences with assessment, and whether you’ve used a method I didn’t mention or one that I did and had success with it.  Start talking!

Alternative Assessment with VoiceThread

Assessment continues to weigh heavily on my mind here at the beginning of the 2010-11 academic year.  While I have worked with students using VoiceThread as an alternative learning artifact, I had not considered using it as form of assessment or digital portfolio until friend and colleague Diane Cordell pointed me to this fantastic post, “VoiceThread as a Digital Portfolio”, in which Chrissy Hellyer, a teacher who was working at an international school in Bangkok at the time of the posting, used VoiceThread as part of student led conferences (you can also read a cross-post of how this teacher implemented VoiceThread here).  In these posts, Hellyer thoughtfully outlines the steps she took before, during, and after the creation of the digital portfolios, including reflections on what she would do differently.

One of my goals this year in my work with our teachers and students, particularly our new cohort of Media 21 students (now called Learning 21), is to help students take a more active role in articulating what they are learning.  After reading this post, I think VoiceThread is a perfect medium for this endeavor; not only could students showcase learning artifacts they feel reflect mastery of course performance standards, but they could also use this digital portfolio to share other key learnings, insights, reflections, and learning tools they feel are an essential part of their story as a learner.

As I start to lay the groundwork to pilot this form of assessment in my collaboration with teachers and students, an additional blog post by Silvia Tolisano, a Technology Integration Facilitator and 21st Century Learning Specialist, is also informing my planning for helping our students create digital learning portfolios.

Are you using VoiceThread or other tools a means of student or learner led assessment in your library environment either with staff or with students?  If so, what advice would you recommend for those embarking on this endeavor, and which web-based tools have you found most effective for creating digital learning portfolios?  I look forward to your sharing your experiences and examples, and I will also be blogging on our own student created portfolios later in the semester!

Google Forms for Assessment, Evaluation, and Reflection

When I think about teaching and learning, assessment is probably the messiest area of the process for me. The longer I teach, the more I seem to struggle with feeling as though I am designing and administering assessment tools that not only measure and reflect the content and skills a student is mastering , but also capture the student’s thinking in the learning process.

One tool that I have found helpful in creating formative and summative assessments this academic school year is Google Forms, a free tool in the suite of Google Docs.  Google Forms allows you to create assessments that can be open-ended or objective in nature; you also can create survey style assessments in which learners respond to questions by ranking or rating their responses.  With Google Forms, you can create assessment questions in the format of:

  • short answer text
  • paragraph or multiple paragraph text
  • multiple choice
  • checkboxes
  • a list of answer choices
  • scaled responses
  • a grid style response

Once you have created your assessment form, you can apply a theme from the menu of choices and then share your survey either via a URL or you can embed into into a web-based tool that accepts HTML code.  Once participants are finished with the assessment, you can easily pull your data into a Google Forms spreadsheet; you can either work with your data within the Google Docs spreadsheet application or you can download into other third-party formats, including Excel and Open Office.  You can also choose to keep this data private or to share it with selected users; for example, when I use Google Forms to engage our Media 21 students in self-assessments, I can easily share the document with Susan Lester, my collaborating teacher, by providing her a private and direct link to the Google Docs spreadsheet that is generated from the data in the form.

Check out how Jessica Hagman, Ohio University librarian, embedded this Google Form for assessment into this LibGuides page:

Here is an example of how I used Google Forms to engage our 10th grade Media 21 students in self-reflection and self-assessment on their most recent presentations:

Google Forms are not just for librarian who work with teen or adult patrons!  The Birmingham (MI) Public Schools’ elementary school librarians Julie Green (author of Super Smart Information Strategies: Write it Down) and Kristin Fontichiaro (editor of 21st-Century Learning in School Libraries) collaborated to create a second grade fixed-schedule unit bringing together the inquiry process as described in Debbie Miller?sTeaching with Intention with age-appropriate resources about seeds, embedded in a wiki.  After their multimedia explorations, students reflected on what they had learned about research and what they had learned about the content area. Google Forms made it easy to integrate the reflective assignment into the wiki space, and the results, gathered behind the scenes in a single spreadsheet, facilitated quick analysis of student responses.

You can see a snippet of their data form that Google Forms pulls into the Google Docs spreadsheet below:

Not only can you use Google Forms for assessing student learning, but you can also utilize Google Forms as a pre-workshop or pre-training tool to assess participants’ prior knowledge, and consequently, incorporate that information into your instructional design.   Polly Farrington recently used this Google Form to administer a pre-assesment to her Tech Camp participants:

In addition, you can use Google Forms to assess participants’ evaluation of your teaching and workshop/training session to improve and craft your practice as a trainer or instructor:

This video, while not geared for a library setting, is a quick and helpful overview of the process of creating and publishing a Google Form for assessment:

Google Forms makes it easy to collect and share qualtiative and quantitative data for evaluating student learning as well as library instruction.  If you are using Google Forms as a means of assessment in your library program, how are you incorporating this evaluation tool?  Please share your best practices here in the comments section of this post.