Continuous Improvement at Industry-Scale

by Aaron E. Silvers (President & Executive Director, The Data Interoperability Standards Consortium)

For 15 years, I’ve participated in the industry of Learning & Development – an industry that touts continuous improvement. There are many books, many conference sessions and workshops and many many blog posts that talk about Continuous improvement as a method of Learning & Development. It’s through practitioners that the capacity for continuous improvement is developed organization by organization.

Some vendors do the work of making themselves accountable to themselves, and ultimately their customers, by their adoption of ISO 9001 or CMMI standards. However, for all the rhetoric about continuous improvement across the industry, for years I’ve seen the gap in the practice of continuous improvement by the Learning & Development industry itself. It’s a long missed opportunity that has undermined Learning & Development efforts across organizations, and it is high time that we do something about it.

xAPI as a technology is finding a lot of adoption, even despite support structures that will unleash its value to individuals and organizations. In the last nine months, we’ve seen conformance requirements and conformance tests developed, and the advent of a formal specification for xAPI Profiles will deliver a much-needed support for semantic interoperability of its data, just as technical interoperability is needed for all the components of enterprise architecture to function correctly. As far as xAPI Profiles go, the machine language translation of ADL’s SCORM Profile has been established and a similar translation of the cmi5 profile is underway. Document revisions are currently happening to make the draft xAPI Profiles specification easier to read, making explicit and implied requirements clear for Learning Record Providers and Learning Record Stores.

All this is well and good and valuable, but for the vast majority of readers and leaders involved with Learning & Development, it means very little. There are more visible, and more pressing issues you’re dealing with daily, and it’s because you can’t get a consistent answer to what should be a simple question: what makes an LMS an LMS? What makes an LRS an LRS?

Improving our definitions

All we can really count on to evaluate such questions on our own are the marketing materials and software trials from different product vendors, and/or the advice of analysts and consultants who make it their business to be familiar with the different systems you might need to depend on in enterprise architecture. It’s from many first-hand experiences that we lack baseline product standards to level-set expectations for what services a product must provide just to be called a “learning management system.” For newer products, like Data Warehouses (or LRSs), Talent Management Systems… the boundaries of such product categories are even more ambiguous.

As a result, we have a market full of enterprise architecture systems — and the only things we can count on them doing are the things we can run software tests for. These are systems that people have to use, and many people who work with data warehouses or learning management systems are not software engineers.

Unless you’re in IT or on the L&D team as an LMS Administrator, the LMS is not the main focus of your job. No matter your technical acumen, if you need to work with a system as part of your job, you need be able to use it. When “good” is so subjective,  the professionals charged with acquisitions can’t know what good looks like. As your capabilities are informed by the quality of your tools (in addition to your skills to use them), it needs to be more transparent to identify which tools have a chance of even minimally meeting your needs.

This is the work that lays ahead for the Data Interoperability Standards Consortium (DISC).

Organizing Stakeholders

At the end of June 2017, DISC will bring together representatives from a number of organizations in the private and public sector, based in the US and internationally, for a one-day summit to get a sense of the issues and concerns these organizations are facing with their adoption of xAPI among other technologies that work with, create and exchange data inside of organizations. Representatives from the vendors who make up DISC’s membership will be among the attendees, presenting enterprise-wide case studies. We’ll then begin our work together as DISC’s team facilitates the collaboration to define what an xAPI-based Enterprise Architecture looks like. I intend to see that this is the first of a series of discussions, in-person and online, with what should be a growing stakeholder membership.

These participating stakeholders have some pretty tangible benefits as a result of their participation. It’s not often that decision makers pursuing similar approaches to developing their infrastructure get the chance to see what others are doing and learn from each other. I anticipate there will be a good deal of “a-has” to be mined from this first meeting alone. To get a chance to sit across from representatives from the vendor community who can digest both their technical and policy concerns about the products in this space, what’s shared will certainly impact work to come.

DISC’s members will gain a lot from that exchange as well. In addition to the opportunity to demonstrate how their products function at scale, they’re going to sit across the table from people who are already adopting xAPI among other data-related technologies, and get a sense for how, together, we can improve LRS products beyond mere conformance to a technical spec. To begin to shape what an LRS product is should prove to be a very exciting and, ultimately rewarding thing.

The benefits, however, don’t stop with just the folks who will be in that meeting room in Santa Clara, California. As with all the projects DISC takes on, we are focused on outcomes that help set up further outcomes.

Boilerplate RFP Language

It may not sound all that sexy, but one major challenge for organizations and vendors alike are ambiguities that are often found in a Request for Proposal. When it comes to learning technologies (maybe HR in general), people writing the RFPs are working off of various sets of requirements, and many times these requirements conflict. Worse, and just as often, the people charged with making the acquisition decisions really don’t know what the technical details really mean. DISC will address this by developing boilerplate RFP language for people adopting enterprise architecture that makes use of xAPI and other technologies. The intention is that by noting in clear, precise language, we can begin to normalize the expectations for what a standard product in the market should do, at a minimum. Adopting such templated language from a trade organization that is concerned first and foremost with the quality of the products on the market in this space, this should help embed better practices for adoption in organizations — particularly around acquisitions.

Market Education

There are three opportunities to develop market education materials that I believe will grow out of this effort.

  • How to achieve “________?” There’s a clear role for DISC to inform people whose work is impacted with the advent of xAPI on what tools help them get into working with xAPI without needing to know everything about the spec.
  • What must IT consider with xAPI? A challenge I expect we must tackle is that each organization has a different approach to enterprise architecture: different stacks and different workflows. DISC comes to this meeting with no expectation that there’s a common pattern, which is a tricky needle to thread. We may well focus on communication: how people engaged in a prospective acquisition get candid answers in terms of how process deployment and implementation work in a stakeholder’s IT stack.
  • What does the future look like? From these discussions, I believe DISC will be able to cultivating different types of stories to paint the possible in several ways which ultimately may inform what product categories need to be defined, as well as help vet the feasibility or practicality of building it.

Product Standards

With our established focus on conformance requirements as well as proposing a functional certification program for LRSs, it should be of little surprise that DISC aims to establish baseline requirements for product categories within enterprise architecture, such as LMSs and Data Warehouses. With standards for product categories, market education efforts to get the word out and boilerplate language for RFPs, we will make it quite easy for decision makers to identify products that meet their basic needs, set apart from products that promise much in their marketing materials but have not been vetted by a third-party to confirm their ability to provide a base set of functionality, let alone deliver system (and semantic) interoperability. This ultimately improves vendor accountability for interoperability by certifying products that meet such requirements. If we’re smart, we’ll make it easy for certified products to differentiate from competing products in the same product category, because the innovations that set them apart will stand out (at first). As other vendors follow suit on such innovations, we can continuously improve those product standards.

That’s how everyone wins. An open spec. A transparent process by which norms are established. A thriving market where getting capital projects to just work isn’t a product differentiator, but the ante to play in the same game.

It’s time we got serious about continuous improvement. Are you ready?


Aaron E. Silvers (President & Executive Director, The Data Interoperability Standards Consortium)

Aaron E. Silvers (@aaronesilvers) has a lethal combination of experience designing and leading development with the big two eLearning standards (SCORM and xAPI), a past life as a K-8 librarian and an M.S. in Curriculum & Instruction from the University of Wisconsin. He has a knack for tackling wicked problems and delivering on big, innovative ideas. Aaron doesn’t just have a passion for learning, collaboration and community building—he lives it—helming multiple communities-of-practice focused on design, technology, and/or learning, including one of the most successful Open Government efforts to-date. As the Executive Director of the Data Interoperability Standards Consortium, he builds partnerships with other trade groups, open source efforts, standards and specification organizations around the world to improve the practice of Learning & Development internationally and across industries, through the development and advancement of technical, product and professional standards and certification programs. As a Partner in MakingBetter, he works with leaders, stakeholders and teams to work more effectively, achieve time and cost savings, and improve the practice of Learning & Development while building the capacity to innovate.

Speak Your Mind

*