No-Reference Quality Metrics: Judging a File With Nothing to Compare It To

Quality measurement splits cleanly in two, and almost everything written about it covers one half.

Full reference compares a file to the original it came from. SSIM, PSNR and VMAF all live here. They are precise, and they are verifiable — feed one two identical files and demand 100 — and they are useless the moment you do not have the original.

No reference judges one file on its own. It is what you need in the situation most people are actually in: a file arrived, something about it is wrong, and the master belongs to someone else.

Why the second is harder than it looks

A full-reference metric can be proven correct against ground truth. A no-reference one cannot, because there is no perfect file to anchor against. Every threshold has to be established empirically — against real examples of the fault, and, the part that gets skipped, against real examples of things that resemble the fault and are not.

That second corpus is where no-reference tools are made or broken. Some measured examples of what its absence produces:

• A backlit sunset with 20% of its pixels blown out, reported as “severely clipped”. It is a deliberate exposure and a good photograph.
• Dense foliage reported as “heavy noise”, because there is no flat area anywhere in the frame and the estimate is taken on the flattest patches available — which are still texture.
• A PNG screenshot told it had “visible JPEG blocking”, because a design system’s 8-pixel-aligned rectangles look exactly like an 8-pixel quantisation grid.
• A letterboxed film clip reported with crushed shadows, because 24% of the frame is the black bars.
• A locked-off shot of something motionless told that “the file claims a frame rate it does not carry” — a causal claim about how the file was made, false, about content the check was never measuring.
• A one-frame video reported as “every frame is a new picture”, computed from zero comparisons.

Every one of those is a number that is literally true and a conclusion it does not support. None of them would be caught by testing against damaged files, because the tool gets damaged files right. They only surface when you test against good files of every awkward kind.

The rules that fall out of it

A check that cannot read a file must say nothing. Not “fine” — nothing. A noise estimate on a pure black rectangle is correct that it is noise-free, and the statement is useless. A focus check on a plain sky has no detail to judge. Declining is a feature, and the report has to say which checks declined, or a missing row reads as a bug.

The headline is the worst finding, never the average. A file that is perfectly exposed, noise-free, correctly encoded and completely out of focus is not 75% good. An average says 75%. The honest answer is that it is unusable.

Across time, though, the median wins. One soft second in a whip pan is motion, not a defect, and worst-takes-all over a whole video condemns nearly everything real. So: worst across dimensions, median across time, with the worst moment reported separately beside the verdict.

There should be no single overall number. Two dimensions measure incomparable things in incomparable units. Adding them produces a figure that looks like a measurement and is really an opinion about weighting. A number per check — anchored so that it can never contradict the verdict beside it — is honest. One number for the file is not.

A measurement is not a finding. Bits per pixel is a ratio against the frame’s dimensions, so a clip of flat colour has a tiny one because the encoder had nothing to spend bits on. Reporting that as “re-export at a higher bitrate” recommends an action that would change nothing.

What stays answerable

Quite a lot, and often the most actionable things:

What does not

Whether an image or video was upscaled. The obvious approach — look for missing high-frequency detail — fails on real content, and not marginally. Film grain and a correctly-exposed night scene both carry more energy in that band than a clean native-resolution shot, because noise is broadband and fills exactly the band the check reads. Round-trip tests are worse: in one measurement the most heavily upscaled file scored highest.

Whether the content is any good. Nothing above is a statement about whether the shot works.

Anything at all about noise in audio. The floor measures beautifully on a synthetic ladder and cannot tell hiss from quiet content. Measured over real material, dynamic range reads 11.8 for music, 6.0 for a field recording, 6.3 for ambience and 3.1 for a limited master, against 17.4 for a genuinely ruined file. Any threshold that catches the ruined one condemns every piece of music in the list.

Cutting a check on measurement, and writing down the numbers that killed it, is among the most useful things a no-reference tool can do — because otherwise somebody rebuilds it on intuition six months later.

Try it on one file

DiffALL runs no-reference checks on images, video and audio — free, no sign-up, and complete rather than a teaser. Each dimension gets its own verdict, its own measurement with units, a 0–100 score within that verdict, and where it applies, a mark on the file showing where the problem is.

And when you do have both files, that is a sharper question and a different tool.