Ten Thousand Books in One Excel File

Paulo Rodrigues5 min read

Ten Thousand Books in One Excel File

A private collector had accumulated a library of over ten thousand books across a residence.

The entire catalogue lived in one Excel file.

That is not a criticism of the owner. A spreadsheet is the correct tool for the first few hundred books, and it keeps being nearly correct for a surprisingly long time. That is exactly how a collection ends up at five figures inside a single sheet.

What a spreadsheet stops doing at that size

Everything except storing.

Searching means Ctrl+F, and Ctrl+F does not know that José and Jose are the same author. There is no reliable way to ask which shelf a book is on, unless somebody typed it into a column and typed it the same way every time. Two people cannot work on it at once. And one careless sort with the columns unlocked can silently misalign ten thousand rows against each other, which is the failure nobody notices until months later.

The ask was a catalogue the owner could actually search, that would keep working, and that would not sit on somebody else's platform.

What we built

A web catalogue, self-hosted in the EU.

LayerChoice
FrontendReact, TypeScript, Vite, Tailwind, TanStack Query, Zod
BackendPocketBase, self-hosted
DeploymentDocker multi-stage to Nginx, on our own VPS
BackupsDaily to object storage, 04:30 UTC

Nineteen catalogue fields per book: title, subtitle, author, publisher, place and date of edition, language, theme and sub-theme, size, pages, binding, collection, translation, illustration, notes, and crucially division and shelf location, because a private library is organised by where things physically are.

Search is accent-insensitive across title, author, publisher and theme. In Portuguese that is the difference between a catalogue people use and one they abandon.

Three permission levels: administrator, editor, and read-only.

What it deliberately does not do

There is no price field. No stock, no cost, no sale, no customer, no order, no invoice, no supplier.

We are listing absences because they are the design. This is a catalogue of a collection somebody owns, not an inventory of goods somebody sells. Building the fields for a shop the owner does not run would have added surface to maintain and questions to answer, in exchange for nothing.

It is worth saying that we got this wrong in our own marketing. Until this week our website described this project as an enterprise library system built for educational institutions. It is neither. We corrected it, in both languages, the day we noticed.

The part that actually matters

Anyone can build a catalogue. The test is whether you can change ten thousand live records without breaking any of them.

Two operations, both on production data.

An import of 29 new books, from a one-off file the owner sent us eight months after launch. Nineteen columns mapping onto the schema. Afterwards we read 551 individual fields back out of the database and compared each against the source: 0 mismatches. A whole-collection diff against a pre-import backup showed 29 records added, 0 pre-existing records modified or removed.

A spelling merge across the collection. Two variants of one division name had drifted apart. We patched 92 records, then verified that exactly 92 were modified, that division was the only field that changed on any of them, and that there were 0 collateral changes. The total stayed put.

Two details about how, which matter more than the numbers:

  • We proved the update semantics on a cloned schema first, rather than assuming a partial update would leave untouched fields alone.
  • The dry run caught a bug in our own import harness that would have inserted all 29 books twice. It never reached the live collection, because we ran it against nothing first.

That is the whole argument. Not "we built a catalogue", but we changed 10,000 live records and can show you, field by field, that we changed only what we meant to.

Where it stands

Live, self-hosted in the EU, backed up daily, and holding over 10,000 books.

If this sounds familiar

The pattern is not really about books. It is a spreadsheet that stopped being a spreadsheet: several people editing it, no validation so a typo surfaces months later, no permissions so everyone sees everything, and search that means opening the file and pressing Ctrl+F.

If there is a sheet somewhere in your business that quietly became the system of record, it is worth twenty minutes of conversation.

This case study is published anonymised. No client, collection or location is named.

Frequently Asked Questions

When should a business move off a spreadsheet?

When more than one person edits it, when a typo can go unnoticed for weeks because nothing validates input, when everyone who needs one row can see every row, or when finding something means opening the file and pressing Ctrl+F. Those are structural limits, not discipline problems.

How do you verify a data migration actually worked?

Read the data back out and compare it against the source field by field, not row by row. Diff the whole collection against a pre-migration backup so you can prove what changed and what did not. Prove your update semantics on a cloned schema before touching live records, and always dry-run the import against nothing first.

Ready to automate your business?

We build AI tools and automation systems for European SMEs — from rapid MVPs to production systems, always GDPR-compliant.

it's human stuff

Weekly AI insights for European SMEs. No hype, just what works.

Keep Reading