It is a service for lawyers and legal professionals in the U.S. The information resources available through Westlaw are sourced from more than 40,000 databases of case law, federal and state regulations, public records, magazine and news articles and more. The training for using the complex Westlaw software commonly used narrated video instruction and side-by-side training — neither of which provide much tracking of the user’s ability to use the software.
To date, approaches to competency-based training applications involved a challenging and large-scale design process and complex workflows to evidence how competency is attained. With the advent of xAPI, however, working with competencies becomes a more interesting project as we slice through the complex workflows. xAPI enabled an approach with Thomson Reuters that turned out to be both simple and powerful.
“Track” the Learner
How exactly do Westlaw users demonstrate competency with this xAPI-based approach? When using the in-line training courseware, learners now perform activities in an overlay directly in the web-based software. Given the material, the activities are primarily simulations in this approach to training. Now, there are many applications that require a software user manual or step-by-step instructions. By moving the instruction inline, a series of guided, contextually relevant step-by-step instructions can essentially became an overlay within Westlaw. More than just text, one can enable a rich learning experience using videos, quizzes, games and more.
To track a learner, we follow a simple but powerful model:
- An Activity leads to an Action or Actions (verb).
- Middleware translate the actions to xAPI Statements and stores them in a Learning Record Store (an LRS).
- The user interface is updated and ready for the software user’s next Action.
It should be noted this is an oversimplification of the process to focus on what the tracking is below.
The xAPI Middleware is made up of an xAPI client application. There are many approaches to developing middleware. The approach we used at Thomson Reuters leveraged a software development kit, or SDK. You can find an example of an open source version of the xAPI client SDK here. SDKs are very common and enable developers to use a technology’s toolset without having to fully understand all of its rules. We developed this xAPI client SDK so it is not necessary to know ALL the business rules to use xAPI. The middleware is an instrumentation between the user’s interaction and the application’s response to that user interaction, taking each action and generating an xAPI statement for it.
Storage of the tracking data happens in two ways: 1) the session data is stored in the browser, relevant in the moment for the activity at-hand, and 2) the LRS storage.
“Listen” for Competency
Having guided the software user through some activities using in Westlaw, enough data is collected to evaluate if they’re performing as is expected. Every web application has some defined state that determines “completion.” By instrumenting to Westlaw’s interface, we were able to track all the software user’s activities up until the application enters this state of completion. By capturing all the actions that a user performs to achieve completion of these event(s), we are able to guage competency.
While the instructional design that goes into building out the rules for a competency based assessment is complex, it is not necessarily part of the xAPI discussion. Creating listeners for the activities that evidence competencies can follow a similar simple but powerful workflow. The goal(s) is/are completed as the learner takes an action. As the action is taken, the xAPI Middleware sends a statement of record to data storage while keeping that action in-session for evaluation of the goal (or micro goal) is completed, the system must queue up next successive goal. If the overall objective has been completed, the competency has been attained If not, competency has not been attained. As it turns out, all of these actions or steps that happen within accomplishing goals or as we call them waypoints, now translate to an activity stream for each learner.
This is where Thomson Reuters realized the real value of xAPI in how it enables recalling historical interaction data. With the ability to store the different states the user is in, Thomson Reuters now also has the ability to determine the state based on the statements within an xAPI stream. It is important to know that an objective is achieved and also how it was achieved. This historical data (summative and formative) can immediately facilitate creating rules around automating interventions and remediations for your learners. It is immediately useful but also, it can be invaluable in the form of formative evaluation data to improve training and for molding how new training happens.
In the case of Thomson Reuters they know how many steps it takes, which waypoints are hit, and how much time it takes for an expert to perform these activities. We can now begin with showing the educators an easy-to-compare-to baseline that shows the steps, the time, the time between steps, and whether the learner achieved the result. This also feeds the learning experience with UI updates. So from the learner’s point of view, the assessment appears as a list with multi-goals that are being checked off for them in the UX as they are proving their competencies.
Thomson Reuters would not be able to do this without something that does what xAPI does. There are unknown formative, summative, strategies and hypotheses that are created dynamically as we gain access to the learning experience data.
Note: The in-line training courseware product leveraged by Thomson Reuters in this case study is a product by Riptide Software called Waypoints. More information can be found on Riptide’s site.
NICK WASHBURN (Director Learning Division, Riptide Software)
Nick Washburn has a 20+ year career working with software companies and hi-tech entrepreneurs in distance learning, and web based software. Currently, Nick is Director of the Riptide Software Learning Division, bringing Riptide Elements® learning products to market. Elements core product development efforts align with the goals of the Advanced Distributed Learning directorate, and Nick is a contributing member of the community, researching, developing and working within the standards for xAPI. During 2009-2012 he led the design and development of an award winning multilingual training product line at a leading training provider for Fortune 50/500 companies. In 2006, Nick led the design and development of the award winning Future Soldier Training System (FSTS) used by the U.S. Army.