DSET Technical Report · DFB TR 2026
DFB: A Soundness Aware, Antiforensics Stratified Benchmark for Digital Forensic Tools and AI Agents
Ankara Üniversitesi · Forensics Research, Ankara, Türkiye
Preprint · 2026 · correspondence: [email protected]
Abstract
Forensic tool evaluation programs are largely static, recall only and predate the arrival of autonomous artificial intelligence in casework. None stratify cases by antiforensics difficulty, and crucially none measure soundness, defined here as a tool's resistance to planted false evidence and its refusal to hallucinate recoveries that are physically impossible. We introduce DFB, a soundness aware, antiforensics stratified benchmark in which forensic software, AI agents and human examiners analyse the same downloadable, fully synthetic artifacts and submit answers to a hidden, server side ground truth grader. We formalise a scoring model that combines difficulty weighted recall, precision style soundness, confidence calibration and an antiforensics resilience curve. We instantiate the benchmark with a single 64 MiB master case, Operation Nightshade, that consolidates twenty one evidence disciplines and one hundred eighty investigation questions into one mountable disk image. The reference solver attains a perfect score, while an identical submission perturbed to fabricate impossible recoveries is measurably penalised, demonstrating that surface accuracy and trustworthiness are separable axes that only soundness aware scoring distinguishes. All artifacts and the scoring interface are public and reproducible.
Keywords: digital forensics, benchmark, antiforensics, soundness, hallucination, AI agents, cross artifact correlation, reproducibility.
1. Introduction
The validation of digital forensic tools rests today on a small number of static reference corpora and challenge sets. These resources verify that a tool parses known structures and recovers known files, but they were designed for a world in which the adversary was passive and the analyst human. Two shifts break that assumption. First, antiforensics has become routine: suspects wipe, alter timestamps, encrypt, hide volumes and, most insidiously, plant false evidence intended to mislead the examiner. Second, autonomous AI agents now perform triage and analysis, and they bring a new failure mode unknown to deterministic tools, namely hallucination, the confident assertion of findings that do not exist. No existing benchmark stratifies cases by antiforensics difficulty, and none measures whether a tool can be deceived. This paper argues that the central difficulty of modern forensics is not recall but soundness, the recovery of truth without being deceived, and presents a benchmark built to measure it.