Flipping the Model: Using data to quantify the informal learning experience

By Megan Torrance, CEO TorranceLearning

The Ann Arbor Hands-On Museum provides a rich STEM-based informal learning experience for visitors with over 250 exhibits in 40,000 square feet of Museum floor, along with classroom, outreach, and distance learning programs. Over 38,000 students in over 675 class field trip groups visit the Museum every year.

Here’s the problem: All too often, the informal learning that takes place on any school field trip doesn’t transfer back to the classroom effectively. Students disperse upon arriving, their activity isn’t recorded, and the connection with curriculum standards is increasingly difficult to establish. This makes it harder and harder for teachers to justify field trips when they’re held to rigorous state-mandated academic standards.

At the same time, the Museum faces several challenges:

  • With an increasingly competitive field trip marketplace, offering a premium experience is a key attractor.
  • Museum floor space is at a premium, yet the demand exists for more exhibits and a more in-depth, fresh experience.
  • Museum staff would benefit greatly from information about how visitors interact with their exhibits in depth.

THE CORPORATE LEARNING SIDE OF THE STORY

And yet, this set of problems isn’t confined to the science museum space. From a corporate learning perspective, online learning has widened the reach and decreased the cost of delivering employee training. When delivered with a one-size-fits-all approach, it often misses its full potential.

Even classroom training experiences aren’t able to meet the unique learning needs of each and every individual in the room. “Event-based” learning models often fail to take into account the on-the-job, in-the-moment needs for training and information: all those little reminders, tips and troubleshooting tricks that we all use to translate what we’ve learned to what we do on the job.

BACK TO THE MUSEUM’S STORY

The Digitally Enhanced Exhibit Program (DEEP) provides engagement and collects data to meet the dual needs of elementary educators and Museum staff. Students on field trips interact with tablets placed at key exhibits. Using beacon name badges to automatically log students into the experience at each tablet, the appropriate grade level (or topic or language) experience is served on the screen. Third grade groups see third grade content, fifth graders interact with fifth grade content, and so on. The tablets guide the students through their interaction with the exhibit, offer challenges, ask questions, and spark reflection and application to new situations. All the while, the tablet records the experience, tracking the specific state science curriculum standards to which students are exposed on every screen, every exhibit tablet, by grade level.

At the end of the field trip, teachers receive detailed reports showing the curriculum standards that were experienced throughout the day. Visits and time spent on popular concepts and popular exhibits are reported before the teacher even gets back on the school bus. Free text and data entered by students at exhibits can be taken back to the classroom for use in other activities. Individual students each receive a personalized report showing the activities they experienced during the day along with their actual answers to questions and challenges posed at each of the exhibits they visited.

The Museum has access to data about the length of time spent at each exhibit, the challenges completed, the data entered by students and the curriculum standards being exposed. Reports can display all data over all time, or hone in to specific school or classroom level reports.

“At the Ann Arbor Hands-On Museum, approximately 13% of our visitors attend on a field trip experience.  That percentage, once higher relative to total Museum attendance, has dropped over the past decade, in part because of budget-squeezed school districts having to reduce field trip spending. DEEP allows us to provide teachers with quantifiable data showing how a field trip to AAHOM exposes students to specific Michigan Science Standards and thus enhances classroom learning (which adheres to the same standards).  This data makes AAHOM stand out above competing field trip alternatives at a time when school administrators have to make tough decisions about where to spend scarce resources.”

Cory Joyrich, Senior Director of Advancement, Ann Arbor Hands-On Museum

DEEP is a novel combination of a number of emerging technologies that offers a simple and elegant solution to the challenge of creating a smart learning environment. At the same time, the architecture of DEEP allows for significant flexibility in hardware and platforms allowing for growth as new technologies emerge.

Beacons for Learner Identification: In the Museum, students are assigned a name badge that includes a beacon with a unique identifier.

 

What’s a beacon?

Beacons are small, often inexpensive devices that enable more accurate location within a narrow range than GPS, cell tower triangulation and Wi-Fi proximity. Beacons transmit small amounts of data via Bluetooth Low Energy (BLE) up to 50 meters, and as a result are often used for indoor location technology, although beacons can be used outside as well. (source: www.webopedia.com)

The DEEP beacon badges are arranged into classroom groups and assigned to a particular curriculum for the day. Currently the Museum offers 3rd, 4th and 5th grade curricula, and a generic Family curriculum, with the ability to expand to additional grade levels, languages and related curricula (History of Science, Inventors & Inventions, Art & Design in Technology, etc.). The beacon badges identify the students as they approach DEEP-enabled exhibits so that the right curriculum can be served up to them and so their activities are tracked by the system.

This is a big deal. With DEEP, we’ve flipped the typical use case for beacons. Beacon technologies are used in marketing and other applications, where the beacons are placed throughout a space and users with downloaded smartphone applications and their Bluetooth service activated receive notifications as they approach a beacon. In the Museum’s situation, it’s not feasible to hand out smartphones to hundreds of elementary students each day. By mounting the expensive technology securely to the wall and putting a $5 beacon on the student, the Museum overcomes this barrier.

Beacons, RFID tags, employee badges and/or other technologies can be used to identify employees as they walk into a room, approach a piece of equipment or move around a location. This goes for corporate, healthcare, manufacturing, military or education applications, Beacon proximity readings can be used to determine how close the person is to the respective target, and who else is present.

In such situations, the goal is for the environment to seamlessly recognize the learner’s presence and capture enough of the right personal information to determine what to do next: whether it’s to offer up training and reminders, serve targeted goal-oriented messaging or even to lock an unqualified individual out of the system.

DEEP is designed to be modular and allow for a variety of identification technologies depending on the installed environment should the Museum wish to replace beacons with another hardware approach.

Tablet-based Engagement Interface: Throughout the Museum, DEEP-enabled exhibits have nearby iPad tablets mounted securely to the wall. Once a student has been identified at the Museum, the tablet greets the student by badge name and the appropriate curriculum is automatically displayed on the screen. Visitors who don’t have a DEEP badge can select a curriculum of their choosing from a START screen. The DEEP screens guide the children toward a more rich and informed interaction with the exhibit itself, posing challenges, asking questions, allowing for reflection and providing additional information. Text and graphics, multiple-choice and multiple-response, free text entry, drag-and-drop and other interactive techniques are used to engage students.

Of course, children move in groups and rarely stand still! The tablets allow for multiple beacons to be recognized and tracked at the same time, following rules to determine which curriculum is displayed if children from multiple groups arrive together. This is something that xAPI enables, but would otherwise not be possible using the SCORM standard. Depending on the type of exhibit and the nature of the interactions, the tablet can “lock in” a badge for an extended period of time, or it can allow the children to be automatically “logged out” when they walk away. The recognized range of the beacon can be tuned define rules for how far away is “walked away.”

In a workplace application, learning and performance support content can be displayed on an adjacent tablet, like the Museum, or embedded into software and screens already on a device. This allows for the immersion of content and experiences within the work environment and a seamless transition between working and learning.

Tracking Activity and Curriculum Standards Using xAPI: We identified very early on that this was an ideal situation for using the emerging xAPI specification for a number of reasons. While on its face this may look like just another fancy elearning application, the complexity and variety of interactions far exceeds the SCORM standard’s capabilities. Whereas SCORM tracks a single logged-in LMS user at a time, DEEP needs to handle multiple visitors in and out of the experience on a question-by-question level and we certainly didn’t want so slow down the children long enough to log in at each station!

DEEP takes advantage of xAPI to track what is otherwise a very informal learning experience at a level of detail that is ground-breaking. At each exhibit, each question is identified by the state science curriculum standards on a grade level basis. DEEP uses xAPI activity statements to record the student’s responses, the correct answer (if one exists), the time spent per question and overall, who else was there at the time, the language used and the curriculum standard by both item number and description. All the data is stored in a learning record store and available for reporting later on.

By using xAPI as the communication specification, DEEP is interoperable with other learning applications both now and into the future. This means that the system can be expanded, or the data gathered from a DEEP experience can be tracked and compared with other datasets. This interoperability and extensibility to future uses was a key factor in the Museum’s decision to pursue the project in this way.

In the workplace learning environment, xAPI data from DEEP can be stored and reported along with data from other learning experiences, providing a rich picture of an employee’s activity. What’s more, xAPI activity recorded by other learning and performance applications could be used by DEEP to offer more personalized content and functionality.

Rich & Detailed Reporting: At the end of the day, as a classroom turns in their DEEP badges and leaves the Museum, staff print a set of reports that provide a rich picture of the students’ visit. Teachers receive a report that shows the time spent and number of interactions with each exhibit and, perhaps more importantly, with each state curriculum standard. Frequently re-visited exhibits – the really popular ones with the kids – are highlighted as well. Students each receive a personalized report that shows details of their day at the Museum, with an actual result from each of the DEEP exhibits they visited and the responses they entered. The Museum will access more detailed ad hoc reporting allowing them to answer questions about the depth of the interactions, the time spent at exhibits and the ways in which different classrooms use their time on their field trips. This rich data will allow the Museum to make future improvements and recommendations.

Behind the scenes, two key functions allow the Museum staff to manage DEEP:

Build & Manage Exhibit Content: The Content Management System is where the Museum’s Education team will be able to update existing content, add new levels of curriculum to existing exhibits, and bring new exhibits online. For each exhibit, interaction rules and beacon ranges are set.

{
  "id": "6759b29a-2e0f-4035-9e5e-a06f85c472dc",
  "stored": "2017-05-23T16:07:59.0772365Z",
  "authority": {
    "account": {
      "homePage": "http://lms.learnshare.com",
      "name": "3444869"
    }
  },
  "actor": {
    "objectType": "Agent",
    "name": "American Bittern",
    "mbox": "mailto:american-bittern@aahom.org"
  },
  "verb": {
    "id": "http://adlnet.gov/expapi/verbs/answered",
    "display": {
      "de-DE": "beantwortete",
      "en-US": "answered",
      "fr-FR": "a répondu",
      "es-ES": "contestó"
    }
  },
  "object": {
    "objectType": "Activity",
    "id": "http://www.aahom.org/exhibits/deep/blockparty/grade3/q6-grade3",
    "definition": {
      "name": {
        "en-US": "Block Party (Grade 3) Question 6"
      },
      "description": {
        "en-US": "Which block pattern did you use to build your wall?"
      },
      "type": "http://adlnet.gov/expapi/activities/cmi.interaction",
      "interactionType": "choice",
      "correctResponsesPattern": [],
      "choices": [
        {
          "id": "answer-1",
          "description": {
            "en-US": "Aligned Blocks"
          }
        },
        {
          "id": "answer-2",
          "description": {
            "en-US": "Offset Blocks"
          }
        }
      ]
    }
  },
  "result": {
    "success": false,
    "completion": false,
    "response": "Aligned Blocks",
    "duration": "PT0M1S"
  },
  "context": {
    "registration": "d81f770e-c4ff-44ba-9b02-1ad40fb60370",
    "instructor": {
      "objectType": "Group",
      "name": "Creamery Elementary",
      "member": [
        {
          "objectType": "Agent",
          "name": "John Doe",
          "mbox": "mailto:deep-exhibits@aahom.org"
        }
      ]
    },
    "team": {
      "objectType": "Group",
      "name": "3",
      "mbox": "mailto:deep-exhibits@aahom.org"
    },
    "contextActivities": {
      "parent": [
        {
          "id": "http://www.aahom.org/exhibits/deep/blockparty/grade3",
          "definition": {
            "name": {
              "en-US": "Block Party (Grade 3)"
            },
            "description": {
              "en-US": "Block Party (Grade 3) exhibit at Ann Arbor Hands-On Museum"
            }
          }
        }
      ],
      "grouping": [
        {
          "id": "http://www.aahom.org/exhibits/deep/blockparty",
          "definition": {
            "name": {
              "en-US": "Block Party"
            },
            "description": {
              "en-US": "Block Party exhibit at Ann Arbor Hands-On Museum"
            }
          }
        },
        {
          "id": "http://www.aahom.org/exhibits/deep/",
          "definition": {
            "name": {
              "en-US": "DEEP"
            },
            "description": {
              "en-US": "Ann Arbor Hands-On Museum Interactive Exhibit Project"
            }
          }
        }
      ],
      "category": [
        {
          "id": "https://www.michigan.gov/documents/mde/K-12_Science_Performance_Expectations_v5_496901_7.pdf#page=15?3-5-ETS1-1",
          "definition": {
            "name": {
              "en-US": "3-5-ETS1-1"
            },
            "description": {
              "en-US": "Define a simple design problem reflecting a need or a want that includes specified criteria for success and constraints on materials, time, or cost."
            }
          }
        },
        {
          "id": "https://www.michigan.gov/documents/mde/K-12_Science_Performance_Expectations_v5_496901_7.pdf#page=15?3-5-ETS1-2",
          "definition": {
            "name": {
              "en-US": "3-5-ETS1-2"
            },
            "description": {
              "en-US": "Generate and compare multiple possible solutions to a problem based on how well each is likely to meet the criteria and constraints of the problem."
            }
          }
        },
        {
          "id": "https://www.michigan.gov/documents/mde/K-12_Science_Performance_Expectations_v5_496901_7.pdf#page=15?3-5-ETS1-3",
          "definition": {
            "name": {
              "en-US": "3-5-ETS1-3"
            },
            "description": {
              "en-US": "Plan and carry out fair tests in which variables are controlled and failure points are considered to identify aspects of a model or prototype that can be improved."
            }
          }
        }
      ],
      "other": [
        {
          "id": "http://www.aahom.org/exhibits/deep/reservation-info/name/american-bittern",
          "definition": {
            "name": {
              "en-US": "American Bittern"
            },
            "description": {
              "en-US": "Name"
            }
          }
        },
        {
          "id": "http://www.aahom.org/exhibits/deep/reservation-info/bundle/birds",
          "definition": {
            "name": {
              "en-US": "Birds"
            },
            "description": {
              "en-US": "Bundle"
            }
          }
        },
        {
          "id": "http://www.aahom.org/exhibits/deep/reservation-info/school/keicher-elementary",
          "definition": {
            "name": {
              "en-US": "Creamery Elementary"
            },
            "description": {
              "en-US": "School"
            }
          }
        },
        {
          "id": "http://www.aahom.org/exhibits/deep/reservation-info/teacher/john-doe",
          "definition": {
            "name": {
              "en-US": "John Doe"
            },
            "description": {
              "en-US": "Teacher"
            }
          }
        },
        {
          "id": "http://www.aahom.org/exhibits/deep/reservation-info/reservationNumber/00000034",
          "definition": {
            "name": {
              "en-US": "00000034"
            },
            "description": {
              "en-US": "ReservationNumber"
            }
          }
        },
        {
          "id": "http://www.aahom.org/exhibits/deep/reservation-info/salesNumber/757750",
          "definition": {
            "name": {
              "en-US": "757750"
            },
            "description": {
              "en-US": "SalesNumber"
            }
         }
        }
      ]
    }
  },
  "timestamp": "2017-05-23T16:07:59.0772365Z"
}

Manage the Visit Experience: Museum staff add field trip reservations and assign curricula to beacon badges, check in badges at the end of a visit, and monitor beacon battery life using the DEEPAdmin tool.

Three unique aspects of DEEP make this a trailblazer product in the learning technologies space:

  1. Flipping the beacon model. Back in the 1980s, visitors to major art museums around the world checked out expensive handheld technology for a self-guided tour in their language of choice. Nowadays many museums offer QR-code or other technologies for visitors to learn more, placing the technology onus on the visitor. Alternatively, barcodes and QR codes on tickets and bracelets require the visitor to pause and scan themselves into the experience before proceeding. With DEEP at the Hands-On Museum, the students have a seamless experience as they move from exhibit to exhibit, without having to bring or carry a piece of expensive technology themselves. Corporate learners keep their hands free for work tasks and don’t interrupt the flow of their work as they go.
  2. Rich, relevant data from experiential learning encounters. DEEP provides educators with useful data about their field trip experience in a way that no other provider in their marketplace does. Details of every exhibit, every student, and every curriculum standard gives teachers information to support transferring their field trip experience back into the classroom. Personalized student reporting helps children share their day with their families and extend the learning to the home. Similarly, in a corporate setting the organization and the learners benefit from the structure that the data provides about learning experiences that in the past could not be tracked at depth or scale as they were beyond the capabilities of SCORM.
  3. Data interoperability with other systems from other vendors. The world of K12 education and the world of corporate learning are each full of fantastic edtech tools, each with their own data sets and dashboards. Our own learning & development industry is itself incredibly siloed! Rather than adding another closed system on a proprietary platform, DEEP leverages the open xAPI specification for communication and data storage & retrieval. This frees up customers to add on components from other vendors and remain portable across platform providers.

In June of 2016, AAHOM tested and evaluated DEEP with select teachers and students on field trips. The response from both was overwhelmingly positive. Of particular significance was the fact that every single teacher thought DEEP would be a useful tool, both to increase the educational value of the experience for their students and to provide data that they could share with administrators at their school.

“This would make it more of an educational field trip.” (4th grade teacher)

“This is pretty revolutionary. Something like this might make me more apt to come here.” (4th grade teacher)

“The principal would love it! Our school is very data driven!” (3rd grade teacher)

 

Speak Your Mind

*