VLA-DSS: a Vision-Language-Action model on the efficiency frontier

Most generalist Vision-Language-Action models answer the question “how capable can we make a robot policy” by adding parameters. Hundreds of millions of them. I wanted to ask the opposite question: how much of that capability actually survives if you shrink the model by an order of magnitude, and where does the capacity have to come from instead?

VLA-DSS is my answer. It is a 28.9M-parameter, RGB-only policy for robot manipulation that runs in real time on weak hardware, and it stays competitive with models several times its size. The whole design rests on one bet: when you cannot afford a large learned vision stack, you replace the missing capacity with structure that is true by construction instead of structure the network has to discover from data.

The problem with scaling a VLA

A standard generalist VLA spends most of its parameters learning to see. It takes raw pixels and, over millions of trajectories, learns features that are stable to the things that should not matter: small shifts, blur, lighting, sensor noise. That works, but it is expensive, and the robustness it buys is only ever empirical. It holds for the corruptions in the training distribution and quietly fails outside it.

If the budget is 28.9M parameters, that road is closed. So VLA-DSS gets its stability from three components that each contribute capability the network would otherwise have to learn.

1. A wavelet-scattering observation encoder

The front end is a wavelet scattering transform, not a learned convolutional stack. Scattering cascades wavelet filters with modulus non-linearities, and its defining property is that the operator is non-expansive. For a scattering operator $S$ and any two inputs $x$ and $y$,

$\lVert S x - S y \rVert \le \lVert x - y \rVert .$

That single inequality is the reason it earns its place. It means the representation is Lipschitz-stable to small deformations: a tiny wobble in the input can only ever produce a tiny change in the features, and that is a guarantee, not a measured average over a test set. The network downstream never has to spend parameters learning to be invariant to jitter, because the encoder cannot amplify it in the first place.

The cost is that scattering features are fixed, so they carry no semantics on their own. That is what the next component is for.

2. Frozen DINOv3 vision, fused with FiLM

For semantics I lean on a frozen DINOv3 backbone. Self-supervised features at that quality would be impossible to train inside a 28.9M budget, so I do not train them. I borrow them, frozen, and pay nothing in trainable parameters for a very strong visual prior.

Language enters through FiLM, feature-wise linear modulation. Rather than concatenating a text embedding and hoping the network learns to attend to it, the instruction directly produces per-channel scale and shift parameters that modulate the vision features:

$\text{FiLM}(h \mid \ell) = \gamma(\ell) \odot h + \beta(\ell),$

where $h$ are visual features and $\gamma, \beta$ are predicted from the language embedding $\ell$. The command does not sit beside the perception, it reaches in and reshapes it. This is a cheap, direct way to make “pick up the red block” and “pick up the bowl” route through the same backbone differently.

3. A Fourier Neural Operator action head

The last unusual choice is the action head. Instead of an MLP or a diffusion head regressing action chunks, VLA-DSS uses a Fourier Neural Operator (FNO).

An FNO parameterizes its transform in the frequency domain. A layer takes a signal $v$, moves it to Fourier space, keeps and reweights only the low modes with a learned complex tensor $R$, and comes back:

$(\mathcal{K} v)(x) = \mathcal{F}^{-1}\big( R \cdot \mathcal{F} v \big)(x).$

Two properties fall out of this for free. The head is band-limited, so it produces smooth action trajectories rather than jittery per-step predictions, which is exactly what you want for a manipulator. And because the operator is defined in function space rather than on a fixed grid, it is resolution-invariant: the action chunk it represents does not depend on the discretization it was trained at. For a policy that has to emit smooth, temporally-coherent motion on limited compute, this is a much better prior than a generic regressor.

Does it actually work?

Yes, and that was the part I was least sure of.

• LIBERO, Object: 79.5% (DAgger, N=200). The auxiliary-x-y variant reaches Object 71% / Spatial 73% / Goal 72% (N=100).
• Robustness comes through. With corruption-augmentation training the policy becomes effectively blur-invariant: on a blur-2 corruption, success goes from 12% to 78%, at a cost of only about 5 percentage points on clean inputs. Noise stays neutral. The structural stability from scattering is doing real work here, not just looking good on paper.
• Efficiency is the headline. At 28.9M parameters it is competitive with Octo-Small (27M) at matched size, and end-to-end it is far smaller than SmolVLA (450M) or an Octo plus T5 stack.

What I take from it

The thesis held up: novel, mathematically-grounded components can buy a tiny model the robustness and resolution-invariance that a big model has to grind out of data. Scattering gives provable stability. A frozen backbone gives semantics for free. An operator-valued head gives smooth, resolution-agnostic actions. None of those cost the parameter budget that learning the same behaviour would.

Efficiency is not a consolation prize for not having a cluster. On the edge, where a robot actually lives, a 28.9M policy that runs in real time and degrades gracefully is the more useful object than a 450M one that cannot.

View the VLA-DSS project detail →