Dawid Dobies / Research / Cognitive load
Research prototype · Eye tracking · Pupillometry

How much you can read from an operator's eyes

An XR headset with eye tracking measures pupil diameter, eyelid opening and gaze direction two hundred times a second. From those three things you can estimate how hard someone's head is working. Below is a working prototype that does it. Drop in your own recording from a Varjo headset, or look at the whole thing on demo data with no hardware at all.

What this is

A workbench, not a product

In the research section I write that I am working on adaptive scenarios that respond to an operator's cognitive load. This is a piece of that workbench, put on display. I am not selling this tool, it is not part of any Varjo offering, and it is not ready to deploy anywhere. I show it because it is easier to judge someone's approach to measurement by looking at the numbers and the formulas than by reading claims.

Nothing leaves your browser. The file you drop in is processed locally, by the code on this page. There is no server, no database, no upload and no storage. There are also no external libraries and no requests to anyone else's domain, the whole page is a single file. Eye tracking data is biometric data under GDPR and I treat it that way here.

Before you believe the numbers. Pupil diameter responds to a change in brightness an order of magnitude more strongly than to mental effort. A bright scene constricts the pupil by millimetres, a hard task dilates it by tenths of a millimetre. If scene luminance changes during a recording, the chart is mostly showing you the lighting.

The index only means something relative to a baseline from the same person in the same session. It does not compare people, it is not a medical device, and it is not suitable for screening anyone for duty or for personnel decisions. The full list of limitations is at the bottom of this page and is worth reading before drawing conclusions.

Prototype

Analysing an eye tracking session

The input is a CSV from eye tracking logging in Varjo Base. If you have no headset to hand, the demo data is a synthetic five minute session with four phases: rest, easy task, hard task, recovery. It goes through exactly the same code as real data.

What is actually going on in the operator's eye. An eighteen second loop: calm sector scanning, a contact appearing, identification and aiming, decision, back to scanning. The dot is gaze, the circles are fixations, the line between them are saccades. On the right, the same quantities the tool below computes: pupil diameter, blink rate and the load index. This is a simulation built to show the method, not a recording of a real exercise. The scene is schematic and does not depict any particular station or vehicle.

Input data

Drop a CSV file here, or click to choose one The file appears in the Videos/Varjo folder next to the .avi recording, once Log eye tracking data is enabled in Varjo Base
No file loaded
How it is calculated

1. Reading the file. A Varjo Base log holds one sample per row, up to two hundred per second. The columns used are left_pupil_diameter_in_mm, right_pupil_diameter_in_mm, left_eye_openness, right_eye_openness, stability, gaze_forward_x/y/z and the timestamps. Sampling rate is derived from the median interval rather than assumed.

2. Blinks. When eyelid opening drops below the threshold and the pupil measurement disappears, the eyes are closed. Closures lasting between 50 and 500 ms count as blinks. Longer ones are treated as the headset coming off or the eyes being held shut, and are excluded.

3. Cleaning the pupil signal. A further 60 ms on each side of every blink is discarded, because during that time the eyelid partly covers the pupil and the reading is too low. Resulting gaps up to 300 ms are linearly interpolated, longer ones stay empty and lower the confidence score for that window. Finally a 100 ms moving average smooths the signal.

4. Saccades and fixations. The angle between consecutive gaze vectors, divided by the time between them, gives angular velocity in degrees per second. Crossing the threshold marks a saccade, a rapid jump of gaze. Stretches below the threshold are fixations.

5. The wavelet index. A simplified version of the Index of Pupillary Activity. The pupil signal is decomposed with a Haar wavelet into frequency bands, local maxima are counted in a low and a high band, and the result is the ratio between those counts. It falls as mental effort rises. The original work by Duchowski and colleagues from 2018 and 2020 uses a Symlet-16 wavelet; this uses Haar for simplicity, so absolute values are not directly comparable with the literature. It is the most experimental component and therefore carries the smallest default weight.

6. The composite index. Each component is converted into standard deviations from the same person's baseline. Pupil dilation and a rise in saccade rate push the score up, so do a fall in blink rate and a fall in the wavelet index. The weighted sum passes through a logistic curve onto a 0 to 100 scale, so the most loaded windows do not all collapse onto the top of the scale. A component that cannot be computed, or that does not vary at all during the baseline, is dropped and the remaining weights are rescaled.

Why the weights are sliders. Different indicators work better for different tasks. Blink rate needs longer windows than the pupil does, simply because blinks are sparse and the count over ten seconds is very noisy. Visual search tasks lean on saccades, memory tasks lean on the pupil. There is no single weighting that is right for everything, and I am not pretending otherwise.

Honestly

What this method cannot do

This is the most important part of the page. Pupillometry is seductive because it produces a good looking chart, and that is exactly why it is easy to overrate.

What follows from this for training

The method earns its place where you compare the same person under controlled conditions: before and after a course, on an easy and a hard scenario, on an old and a new station interface. There the difference is interpretable. It has no place where someone wants to use it to grade an individual soldier or decide on their posting.

If you wanted to use something like this in a study on human participants, you need ethics committee approval and informed consent. That is not a formality to skip, because eye tracking data is biometric data.

Where this comes from

The prototype came out of preparation for doctoral research on adaptive tactical scenarios in XR. The direction and the list of texts in preparation are described in the research section. None of it is published yet and I do not present it as a track record.

Let's talk about measuring effectiveness Training cost calculator