A Plan to Solve Motion Sickness in VR
Motion sickness is the single biggest bottleneck for virtual reality as an entertainment product.
The reason is simple:
Around half of the population will get motion sickness.
And, when it happens, it’s strong enough to instantly make every other aspect of the technology irrelevant.
Put simply, you might be exploring the most amazing virtual world you’ve ever seen, but you’ll still take the headset off the moment you feel like throwing up. It’s a total dealbreaker.
So, if we want virtual reality to seriously compete with other forms of entertainment, we need to solve motion sickness.
(And it’s also a problem for first-person view drones and any kind of virtual reality simulation where the user moves in the virtual world, like flying or combat simulators)
What causes motion sickness?
We still don’t understand everything about motion sickness, but there’s a lot of evidence that it’s ultimately caused by:
Seeing yourself move (on the screen)
But not feeling yourself move with your inner ear
In other words, motion sickness happens when we press a button to move in a virtual reality game, but don’t move at all in real life.
How can we solve motion sickness?
So, to prevent motion sickness, we need to either:
Stop us from seeing ourselves move
Make sure our inner ear feels whatever movement we’re doing in the virtual world
Now it’s possible to mess with our inner ears with some sort of extra hardware or motion sickness pills. But extra hardware adds cost and inconvenience, and pills come with side effects. So I think it’s better to focus on what we’re seeing instead.
Also, we already have free access to the user’s eyes, so any visual solution we come up with could more-or-less instantly scale to the entire population of virtual reality users.
How do we see ourselves move?
To see where we’re moving, our brains have to do a lot of processing with very little time and resources.
That’s because, to not fall down and die, we have to make adjustments within tens of milliseconds. And we have to do so with our entire brain having access to 50x less power than a toaster.
This means that our brain can’t possibly take into account every tiny detail of the “video feed” we’re seeing. It can only process the “video features” that are most useful to seeing where we’re moving.
So what are these features? While we don’t know 100%, decades of neuroscience research point to things like:
Brightness contrasts, not color contrasts
Big things, not small things (low spatial frequency)
Fast things, not slow things (high temporal frequency)
Things in our peripheral vision, not our central vision
Particular patterns of flow, like a bunch of things all moving left when we’re moving right
And this makes sense because, say, small branches might move together in random directions thanks to the wind, but it’s pretty unlikely that a bunch of big trees move together unless we’re the ones moving.
So there’s no point in considering every single one of those little branches (and likely getting confused by them), when we can make a reliable calculation with just a few big trees. So that’s probably what our brains have learned to do.
How do we stop us from seeing ourselves move?
Given all of this, we basically have two main goals when trying to reduce motion sickness in virtual reality:
Remove video features that our brains use for processing self-motion
But do so with minimal visual disruption, so that we don’t ruin the experience for the user
In other words, we’re trying to selectively disrupt self-motion processing, while keeping other aspects of visual processing untouched.
And this seems plausible, since we can already trigger motion sickness with visual changes that are barely noticeable (like shaking the user’s view by very small amounts).
So I don’t think it’s insane to suggest we can do the opposite too.
What are the existing solutions?
The current solutions to motion sickness were largely discovered through manual trial-and-error, rather than some sort of rigorous optimization process. So they remain quite primitive.
For example, the most popular technique involves just completely blocking off the user’s peripheral vision every time they move in the virtual world.
This is simple to implement and reduces motion sickness quite a bit, and so it’s used in basically every virtual reality game.
However, many users will still eventually get sick after 30-60 minutes and, more importantly, blocking off half the screen is very disruptive to the experience.
And that disruption really matters, since the entire promise of virtual reality is that it makes you feel like you’re really inside a virtual world.
If you’re blocking off half the screen, it no longer achieves much beyond what a normal computer screen already does. Why use virtual reality at that point?
And all other techniques suffer from the same problem: they simply cause too much visual disruption for too little reduction in motion sickness. They just aren’t selective enough.
Why is it so difficult to come up with solutions?
But why is this? Smart people have tried to solve the problem for decades, so why are we still using a technique as primitive as just blocking off half the screen?
There are three main reasons why it’s so hard to find better techniques:
Our motion processing system is very hard to disrupt, since it’s a critical system that evolution has tinkered with for hundreds of millions of years. We can’t just remove some range of, say, brightness data and call it a day (I’ve tried). To succeed, we’ll have to create stimuli that our brains haven’t learned to handle in those millions of years. And that probably means messing with a bunch of different video features dynamically and all at once.
There are an unbelievable number of things to try, since we’re tweaking millions of pixels of video data and our eyes are good enough that a lot of those tweaks might reasonably have an impact.
And, lastly, it’s very hard to quickly find out if some tweak actually worked. Motion sickness only becomes noticeable after minutes of exposure and there’s a lot of variance from day-to-day and person-to-person. So it takes lots of participants and weeks of testing just to see if one tweak had an effect. Researchers will also get used to motion sickness over time, so they can’t even test tweaks on themselves.
In other words, there’s a lot of stuff to try, most stuff won’t work, and it’s very hard to see if that stuff actually works.
Our research process is absurdly inefficient, and that’s why our techniques still seem so primitive.
What can we do about it?
Is there some way we could speed up this process? I think so. And by a lot, thanks to recent advancements in machine learning.
Here’s an example of what we could do:
Find or create a dataset of videos labelled for how much motion sickness each frame causes, with associated headset position data (since we only care about on-screen movements that the user didn’t make in real life)
Use that dataset to train a machine learning model that predicts a motion sickness score for each frame (a motion sickness discriminator)
Get an existing machine learning model that predicts whether humans will see visual artefacts in video (a visual artefact discriminator) or, alternatively, use a series of heursitics based on what we know about human vision
And then run an adversarial attack process, where we do a bajillion little tweaks on each video to see which tweaks minimize both (a) the sickness score values of our motion sickness discriminator and (b) the visual artefact scores of our visual artefact discriminator.
After this process, we’ll ideally have tweaked videos that more-or-less invisibly prevent us from seeing ourselves move.
If this works, we can then develop a model or algorithm to apply these tweaks on top of the video feed that users see while playing a game or watching a movie in virtual reality.
The core idea here is that it’s millions of times faster to test tweaks with a machine learning model than with humans. We’d be able to make months of progress in seconds, at least in theory.
But, of course, this massive speedup doesn’t come for free. We’ll have to do a lot of work to make sure that the model is airtight and that our tweaks don’t just affect the model without actually affecting humans.
That’s no easy task, but it still seems much easier than testing every possible tweak with humans.
The progress so far
As far as I know, there are no openly accessible motion sickness datasets, nor machine learning models that predict motion sickness from video data alone.
Since I don’t have the resources to create a large dataset and train my own model, I’ve instead decided to validate the approach with existing machine learning models of human motion processing (as opposed to motion sickness).
The reason why these exist is that it’s much, much easier to get data on which way humans think an object is moving than it is to measure motion sickness. We’re talking seconds versus hours per trial.
So my first attempt was using this motion processing model by Sun et al. (2025). It takes in a video and tries to predict which way a human would think stuff is moving in that video — and it does so very well, even mimicking many of the mistakes that humans make.
So I tried out an adversarial attack on this model, using something called Projected Gradient Descent. Here’s how it works:
Try a bajillion tweaks to a video
See which tweaks cause the model to underestimate how much stuff in the video is moving
Make those tweaks
And then try another bajillion tweaks and repeat the process.
However, since no model will ever fully replicate human perception and there are so many potential pixels to change, it’s basically inevitable that this attack will find some magical combination of a few pixels that breaks the model but has no chance of working on humans.
So I did two main things to mitigate this:
Limit tweaks to large changes that sit within the preferred spatial frequency range of human motion processing (around 0.5-1.5 cycles per degree, according to neuroscience literature). I did this by making tweaks at a low resolution and then later upsampling to the actual video size.
Make it so that any tweak is always tested on multiple variants of the video frame, each with some sort of random transformation or noise added onto it (i.e., Expectation Over Transformation). This ensures that tweaks don’t just perfectly mess with the set of pixels in this one video frame.
Then, to make it so that tweaks aren’t too visually disruptive, I also prevented them from changing the RGB value of any pixel by too much.
So, did it work? Well, let’s go through some examples.
Here’s a video with no tweaks done to it:
And here’s what it looks like after I ran this attack and found tweaks that stopped the model from seeing movement in the video:
Here’s are just the tweaks themselves, which are overlayed on top of the video:
That’s not too compelling yet, so here’s what it looks like when we increase the minimum size of the tweaks:
Here’s what happens when we allow the tweaks to change RGB values more (while reverting back to the decreased minimum size):
It still definitely isn’t enough to mess with which way we think objects are moving, but the trees seem to start jumping around a bit (at least for some people I tested with).
The reason is likely that these tweaks happen to be at a similar spatial frequency and brightness level to the trees, but not other objects in the video.
So here’s what happens when we force the tweaks to at least somewhat match the spatial frequency of the background:
It’s still not that compelling. And the reason is probably that we’re running up against the underlying problem with attacking this model: it only tries to predict how humans see little parts of the image move.
It therefore has no concept of how humans will see the entire scene moving, which is what we’d care about when trying to prevent humans from seeing themselves move.
Nevertheless, we can cheat a bit and force the tweaks to change the motion of the entire scene instead.
At this point, we’re hard-coding a lot of assumptions and kind of eliminating the point of attacking a machine learning model. But it’s still interesting to see what happens.
So here’s what it looks like when we make our attack warp a fixed noise texture, rather than changing individual pixels:
Here’s what that tweak looks like in isolation (note how its moving and warping a noise texture, rather than changing individual pixels):
Here’s that tweak when we let it change RGB values more:
Here’s a tweak with a different method of matching the scene’s spatial frequency:
And here’s that tweak but when we let it change RGB values more:
And, just for fun, here’s a tweak when we make the tweaks slide the video’s existing pixels, rather than warping a texture:
And here are some tweaks on a video where we’re rotating, rather than shifting to the side (it’s seems much less effective):
So does it do anything for motion sickness? I haven’t tested it thoroughly, but I highly doubt it.
One problem is that we’re clearly still affecting the model, while only affecting humans a bit.
So we’ll need to come up with more ways to ensure attacks transfer to humans too, like attacking multiple different machine learning models with different architectures all at once.
That way, our attack is less likely to be super specific to just one model and is much more likely to transfer to the “model” we care about (i.e., real human motion perception).
There’s also the fact that, again, this model predicts how humans will see things in the scene moving. It doesn’t even try to predict how we see the entire scene moving or, more importantly, what makes us feel motion sickness.
So, even if we assume that the model perfectly replicates human perception, attacking it will only create tweaks that mess with how we see objects moving in the video. It won’t do anything for how we see the entire scene moving.
All that being said, these initial attacks did still create some weird and somewhat confusing effects, especially for detailed (high spatial frequency) regions with side-to-side movement.
So it’s clear that there’s at least some degree of transfer to humans. I just need to improve the process and attack a model that better reflects how we see our own movement.
Next steps
I only recently found out that people have already trained machine learning models that predict which way humans think they’re moving when looking at a video.
This one by Mineault et al. (2021) seems interesting, for example.
I’ll attack one of these and, if results seem promising, run a larger study to see if these attacks also reduce motion sickness in humans.
Here’s an early (and not very effective) test with Mineault’s model:
If attacking this sort of model still doesn’t yield anything promising, I’ll create a dataset (or try to get access to an existing one) and train a model that explicitly predicts motion sickness from video data. That’s a much bigger effort, though.
