Building Local AI Search

Bring the search to the library

A local photo library is personal, messy, and often much larger than its owner remembers. Search should make that collection easier to use without making an upload the price of entry.

The starting point is a pipeline with two distinct jobs: preparing the library and answering a query.

Build the index once

For each image, a model produces a numerical representation, often called an embedding. Store that representation alongside a reference to the original file.

# An illustrative pipeline, not LAVU's production implementation.
for photo in library:
    if index.needs_update(photo):
        vector = model.encode(photo)
        index.save(photo.path, vector)

The expensive preparation should not repeat every time someone opens the app. A useful index tracks which files are new or changed.

Keep the query path short

Represent the query with a compatible model, compare it with the stored vectors, and return the closest matches. A common starting point for comparison is cosine similarity.

Local photos → Model → Search index
                         ↑
Query image → Model → Similarity search → Results

The interface is part of the system

Model quality matters, but it is not the whole experience. A person also needs to know whether indexing is still running, which folders are included, and how to get from a result to the original photo.

Useful questions include:

• Can indexing stop and resume?
• What happens when a file moves or disappears?
• Can a result lead to another search?
• Is the difference between indexing and searching clear?

Start with a complete path

Before optimizing individual stages, get a small collection through the entire workflow. Import, index, search, open the original. Then test with a library that looks like something a person actually keeps.

This is a sample technical note. It describes a general approach, not a benchmark or a claim about released LAVU behavior.