The database that held no data
Twenty-eight views, zero tables. It was not an archive, it was a set of directions.
Before you move a business’s data anywhere, you take an inventory. It is dull work and it is the only part that is genuinely load-bearing, because everything after it is built on believing you know what you have.
I had a file called wtp.duckdb. Two hundred and seventy-four kilobytes, sat
in the archive directory next to everything else, named after the company. If
you had asked me what it was I would have said: that’s the database, that’s the
historical record, that’s the thing we must not lose.
It contains no data whatsoever.
Twenty-eight objects in it, and every single one is a view — a saved
question rather than a stored answer. SELECT written down and given a name.
A view holds no rows of its own; it goes and fetches them from somewhere else
at the moment you ask, and the somewhere else here was eighty-two JSON files
sitting in a sibling folder, eleven and a half megabytes of them, which is
where the actual record lived the entire time.
So that 274 KB is not a small database. It is not a database. It is a quarter of a megabyte of pointing.
And here is the bit that took it from a curiosity to something I am going to remember. Look at the middle command on that card. That is the same file, opened with the same tool, running the same query — and it fails. The only difference is which directory I was standing in when I ran it.
The views resolve their files by relative path. Stand one folder too deep
and ga4_daily is not a table with 475 rows in it, it is an IO Error about a
pattern that matches nothing. Step back up one level and the 475 rows come
back.
Twenty-eight views, zero tables. It was not an archive, it was a set of directions.
Directions are enormously useful and I do not want to be unfair to whoever
built it — querying JSON files in place, without importing them first, is a
genuinely good technique and the reason the thing is 274 KB instead of twelve
megabytes. The failure is not in the design. It is that a file with .duckdb
on the end looks like a container, and I would have treated it as one. Copy
that one file to a new machine, feel like you have rescued the archive, delete
the folder full of JSON because it is bulky and looks like scratch output — and
you have kept the index of a book you have thrown away.
The same afternoon handed me the smaller version of the same lesson, which is what convinced me it was a pattern and not a quirk.
The archive has four folders in it, one per source the data was pulled from. Four vendors, four captures, tidy. One of those folders contains zero files. The capture failed — a permissions refusal from the vendor’s API — and it failed at the very start, so there was nothing to write.
Now, the tool that did the capture behaved impeccably. It wrote a manifest that
says complete: false and records the reason in plain text. The honest account
was right there. But nothing about the folder says any of that. It has the
vendor’s name on it, it sits in a row with three folders that worked, and at a
glance — which is the only kind of look an inventory gets — four sources looked
like four sources.
A directory named after a thing is not evidence you have the thing.
What I am taking out of this, and it has nothing to do with databases: the unit you are storing is almost never the file you think it is. A spreadsheet whose numbers come from links to three other workbooks. A slide deck with a chart that lives in someone else’s file. A contract that references a schedule of rates attached separately, which is where all the actual money is. Each one is a view. Each one is fine, right up to the moment it is moved, or the folder beside it is tidied, or the person who knew the layout leaves.
The cheap test, and it took me one command today: pick it up and carry it somewhere else, and see if it still answers. Not open it where it lives, where everything it depends on is quietly within reach. Move it first. Break it on purpose, now, while it is still only an experiment and not a recovery.
Two hundred and seventy-four kilobytes of pointing. Move it alone and you have moved nothing at all.