How semantic photo search works

Semantic search matches what an image is about, rather than the letters in its filename. You type a phrase like 'two people laughing at a table' and you get photos that show that, even if they are named IMG_4821 and sit in a folder called 2021_import.

Older photo search relied on filenames, dates, or tags you added by hand. Semantic search removes that setup, because the meaning is read straight from the picture.

An embedding is a vector, a list of numbers, that captures the content of an image. Pictures with similar content end up with vectors that sit close together. PhotoCore stores a 768 dimensional vector per image in a small local database.

Text gets the same treatment. The model places your words in the same space as the images, so a phrase and a photo can be compared directly.

When you search, your phrase becomes a vector and PhotoCore compares it to every image vector using cosine similarity, which measures the angle between two vectors. A smaller angle means a closer match, and the top results are ranked by that score.

On Apple Silicon this comparison runs through Accelerate.vDSP, so a search across 10,000 images returns in under 50ms.

The model maps text and images into one shared space, and it was trained on many languages. That is why a query in English, Italian, Spanish, Japanese or more than 100 other languages lands on the same photo, with no setup or translation step.

Yes. The whole pipeline can run on device. PhotoCore uses SigLIP 2 through CoreML on the Apple Neural Engine to build the vectors, and keeps them in a local SQLite cache and in XMP metadata.

• No image is uploaded to any server
• Search works with no internet connection
• The index lives on your Mac, under your control