Wearable Technologies: The Validation Journey

Featured story
'The devil is in the detail in terms whether a device is appropriate for use from scientific, technical, health and security perspectives'. David Hardman, Senior Research Analyst, and Dan Basacik, Lead Human Factors Specialist.

Wearables are not all at the same readiness level for rail deployment. For example, devices for tracking lone workers generally comply with established standards, while some diabetes monitoring wearables are still at an early development stage. In this article we look at the “validation journey” wearables need to undertake if they are to become part of an approach to managing safety and health, as well as at the assurance process the rail industry should consider.

The validation journey for wearables

Figure 1: the validation journey for wearables

Scientific validation: the case of sleep trackers and fatigue monitoring devices

To begin with, the accuracy of any wearable should be assessed before it is deployed, particularly because many wearables provide estimates based on indirect measures. For example, wrist-worn sleep trackers apply algorithms to the output from an accelerometer, which measures the movement and orientation of the device. Because the different stages of sleep are associated with differing degrees of bodily movement, trackers use the movement data to estimate the time spent in the different stages of sleep, as well as the total amount of sleep in a night.

Research from sleep laboratories shows that wrist-worn sleep trackers are reasonably accurate for assessing the sleep time of non-insomniac individuals, albeit with a tendency towards overestimation (a bias that can potentially be corrected for). Whether this is sufficient evidence of accuracy depends on what the device will be used for. Some wrist-worn sleep trackers may be accurate enough to give researchers an idea of the average sleeping habits of a group of ‘normal’ sleepers. However, they may not be accurate enough to be used to diagnose a sleep disorder.

Another important consideration is the scope of what the tool is measuring. Fatigue monitoring devices are a case in point. Fatigue is defined by the ORR as “a state of perceived weariness that can result from prolonged working, heavy workload, insufficient rest and inadequate sleep”. This means fatigue has a broad scope and can manifest itself in different ways. Most devices estimate sleepiness/alertness from measurements of eye movements, the brain’s electrical activity or the user’s heart rate. Thus, none of these devices monitors fatigue directly and none addresses the full scope of fatigue.

Most devices will, at some point, get it wrong. The challenge for any manufacturer is to optimise their product so that it picks up on genuine episodes of fatigue, doesn’t remain silent if the wearer is drifting into a microsleep, and doesn’t raise an alarm when the wearer is alert. False alarms can cause distractions and lead to loss of trust in the device.

These issues show that expert knowledge is needed for the scientific validation of wearables, and that each device needs to be assessed individually, as often their underpinning approach and algorithms differ. In addition, for devices that are intended to facilitate behaviour change, validation should include an examination of whether that change actually occurs. As there have been relatively few studies so far, more research is needed on these fronts.

Technical reliability: the case of lone worker devices

Wearables are basically body-worn sensors with wireless communications. They use short radio frequencies and other wireless protocols which may encounter or provoke some interference. This is especially true on the present rail network, where mobile signal can often can be lost. This may be a critical matter for lone workers in need of constant signal coverage.

Electromagnetic interference is another phenomenon which can degrade the operation of wearables. Electromagnetic compatibility is regulated by several national and international standards, to assure that the wearable does not interfere with other devices and can function normally in environments where there is electromagnetic radiation. As a general rule, error correction and fault tolerance must be used to ensure reliable operation in harsh environments.

The need to ensure continuous operation in safety critical contexts may mean that some devices should follow standards that are more demanding than for most consumer wearables. This might means trading off “wearability constraints” (i.e. size, weight, placement, low power, compatibility with PPE) against robustness and ruggedness.

Health risk assessment: the case of smart glasses

Wearables should not expose their users to health hazards, whether by their normal utilisation or by operating failures, breakage or error. There are already standards for electrical safety (user protected against electrical shock or burns when overheating), chemical safety (lead, nickel, cadmium), Specific Absorption Rate (SAR), that is amount of electromagnetic energy generated by the device which is absorbed by the body.

“Biocompatibility”, or the absence of harmful effects of long term use, should also be assessed. For instance, in the case of smart glasses some academics and technology testers have expressed concerns about both the impact on wearers’ eyesight, and the potential effect of radiation exposure. Under US law manufacturers are required to test and publish the SAR for body-worn devices that emit microwave radiation. No personal wireless device may have an SAR that exceeds 1.6 watts/kilogram (W/kg). Although the versions of Google Glass that were considered for commercialisation in 2014 have an SAR of 1.42W/kg (which compares with that for an Apple iphone X of 1.17W/kg),simple compliance to the SAR standard did not take into account that smart glasses were designed to be worn on the head for much longer periods than a smartphone.

These examples suggest that wearables should undergo a health risk assessment which also takes into account their expected use as an integrated system, beyond existing standards for their components. Lists of assessed devices should be kept at an industry level, like the lists that the medical sector keeps for medical devices.

Secure by design?

Wearable electronics are a subset of the Internet of Things (IOT). They are much more vulnerable to cyber attacks than traditional LANs and Wireless networks, because IOT lacks secure protocols. According to security researchers Scott Erven and Mark Collao, around 68,000 medical devices at an unnamed US health group were found to be available for hackers to access online. This has prompted the information security industry to call for medical device manufacturers to factor in security at the design stage and build cyber security into all internet-enabled devices. If the industry is to embrace wearables the same robust approach must be applied, and cyber security built in to all industry devices and all wearables.

Data privacy

Many laws worldwide regulate the use of personal information, but few holistically address the collection, storage, use, sharing and disclosure of personal information obtained through wearables. In the UK, organisations must comply with GDPR and the 2018 Data Protection Act and we explore what this means for wearables in a previous blog.

Industry agreement on usage and scope

Limiting the scope of application of wearables is key for ensuring their ethical use: it is tempting to want to exploit to the maximum the data collected by wearables, but wearables should not become multipurpose spying devices, nor should wearables that were validated for a specific utilisation and context be overly trusted when used differently than for their original purpose. This is especially important in the case of disciplinary or legal proceedings.

An industry-wide understanding of the correct use of wearable devices would ensure they are regarded as indicators and enhancers rather than as substitutive systems which are overly relied upon. Also, any potential positive or negative impacts of wearables on users’ behaviour is something that needs consideration.

Which assurance scheme?

Although regulations and standards specific to wearables are under development in certain jurisdictions, there are few standards that cover wearables per se. Regulators apply requirements at the component level, based on standards that are not specific to wearables (e.g. electronics).

An industry level assurance scheme would likely follow the “validation journey” described in this article, and would also alleviate data privacy concerns.

Ultimately, those introducing a wearable would need to demonstrate a thorough understanding of how it will be used in the rail context. To that effect, a similar approach to the one given in Contrasting paradigms forthe development of wearable computers (Baber, Haniff and Woolley, IBM Systems Journal) would be valuable:

  • Requirements specification (wearable generic and application specific)
  • Scenario generation (context of use)
  • Option reduction (eliminating certain designs based on pros and cons)
  • Design/evaluation (working prototypes evaluated by end users based on agreed criteria and performance levels).

Where does this leave organisations looking to purchase wearables?

This article has highlighted that the devil is in the detail in terms whether a device is appropriate for use from scientific, technical, health and security perspectives. In the short term potential buyers should look beyond the marketing literature and seek high quality evidence from credible sources and see how this matches their specific needs and opportunities.

In the longer-term we foresee a simpler world for organisations looking to purchase wearables in which much of this work is done by assurance schemes, creating the reassurance and appropriate guidance the industry already benefits from in more traditional aspects of its work. However, it will always be the job of individual organisations to decide who should use any device, what they should use it for, when and how they should use it.

Haven’t found what you’re looking for?
Get in touch with our expert for more information
David Hardman and Dan Basaick
David Hardman & Dan Basacik