Project Everything

The tools, collected and tested

Claude Code roles studied the candidates per layer for size, memory, licence and source, then measured the strongest ones. Everything runs locally on one RTX 5070 Ti Laptop GPU with about 10 GB usable, on PyTorch, ONNX Runtime, transformers and llama.cpp, with SQLite files joined by checksum.

Documents. Tesseract with German and English models, OpenCV to straighten and binarise scans, and own readers for PDF text layers, Office formats, HTML and mail files, plus Word and Excel as separate processes for old binary formats. A gate gives each page one of three outcomes: ok, uncertain or review. A page that stays poor says so.

Files and metadata. An own file layer with checksums, Pillow and pillow-heif, ExifTool 13.59 and ffprobe. Own registers read mail, calendar and bookmark exports.

Audio. faster-whisper large-v3 behind the same gate, because a transcript that sounds equally sure everywhere is more dangerous than one that shows where it guesses.

Images. pHash and dHash for near duplicates, SigLIP 2 so400m as a multilingual search vector, RT-DETR R50 for finding people, a structural splitter for collages, and Qwen3.5-9B through llama.cpp for descriptions. Qwen3-VL-8B was the comparison, Gemma 4 and Florence-2 stayed on paper.

Faces. Five detectors (YOLOv8x-Face, RetinaFace R50, SCRFD 10G and 34G, YuNet), thirteen recognisers (AdaFace IR-101, TopoFR R100, ArcFace R50 and R100, TransFace-B, LVFace T, S, B and L, EdgeFace-Base, MagFace R100, CR-FIQA R100, SFace), and CR-FIQA and MagFace as quality scores. They were measured on a test set of 8,389 images, and standard collections such as WIDER FACE, LFW, AgeDB, FDDB, TinyFace and FG-NET were gathered beside it.

What the tests showed:

• Worked: the document and speech gates. Doubtful text is marked instead of looking as sure as the rest.
• Worked: SigLIP 2 at about 30 ms per image, with German search directly. On paper OpenCLIP was English only and jina-clip-v2 had the tighter licence.
• Worked: Qwen3.5-9B at 3.5 to 5 s per image. One model is enough, and it is the bottleneck of the chain.
• Worked: AdaFace IR-101 as recogniser, 97.5 % at a false accept rate of 1e-3, ahead of TopoFR at 96.3 % and ArcFace R50 at 95.8 %. LVFace-B, the leader on paper, reached 93.4 % here.
• Worked: yolov8x-face as the most balanced detector, at 98.4 and 98.2 % on two sets. "Right" here means three of five detectors agreed, so 1,162 cases stay open.
• Worked: CR-FIQA for quality. Dropping the worst 10 % cut false rejection from 4.5 to 2.0 %.
• Partly: grouping. A threshold of 0.50 split one identity into 6.6 groups, and 0.25 with AdaFace finds 97.6 % of it.
• Partly: Tesseract on photos. Automatic page analysis is unusable there, so photos go through mode psm 11, and even that adds little outside screenshots and work images.
• Did not work: SFace, the free licence candidate, at 69.8 %. Most strong face weights are non-commercial even when the code is MIT.
• Did not work: combining AdaFace with ArcFace, 97.1 against 97.5 %.
• Did not work: age and gender estimates, unusable in measurement and dropped from the chain.

Still only planned: Opus-MT on ctranslate2 for translation and ffmpeg keyframes on every scene change, at most 40 per video. Sound events, speaker separation and a music fingerprint are not chosen.

The road so far

A register comes first: it records what exists, what it belongs to and what points at it, so a lost state can be rebuilt instead of guessed.

Documents come next. They went through text reading and OCR, and a first index came out of that, with the checksum as its key rather than the path. Text inside photos is only partly done, and foreign language pages need a second reading with the right language.

Photos and audio have test results. Measuring runs on 150 images gave 3.6 to 5 s per image and showed that the description sets the pace. Transcription has run on a first part of the audio. Video has only a first test: an image sequence from a film, described by five local models.

The hard part of this road is scale. The whole chain took 7.6 to 25 s per image in tests, and its slower paths, at about 16 to 25 s, extrapolate to 870 to 1,404 hours for 200,000 images on one laptop GPU, so every extra layer has to justify itself. A run of that size also has to survive restarts, keep results per file and repeat nothing, which is why the key is the content and duplicates are computed only once.

Claude Code agents planned, coded and ran all of this, as separate roles with one single writer for the register. I set the vision, made the scope decisions and gave each go.

What is still missing

Almost everything that would make it a web.

• A system to connect. The concept names layers and keys, and a first tool lists people, organisations and places with their mentions. It keeps every mention under its raw name even when nobody can assign it yet, and it merges nothing only because it looks alike. But knots are not yet threads: one identity across face, name in a document, voice and sender is an open schema question that has to be settled before the big run.
• A system to visualise. Person gallery, map, timeline and search by words exist as a plan. At the very end the web has to become something you can actually see.
• The big runs. Photos and video have only had tests, and a trial of 500 images and 20 videos has to come before models and schema are frozen.
• The open sound layers.

No part is finished as a product. This is an early concept with a tested toolbox.

Connected

• Masterface
• X-Ray