Cleanly fall back if keyed filters cannot be converted from expected to actual

E.g. if actual is an unsigned4, and expected is an unsigned2 a keyed filter cannot sensibly be converted back to work on the actual file.