How I Cut 60% of Indexing in My Personal Screenshot Library
After 300+ days and hundreds of gigabytes of local screenshots, you want to build your own search engine? I tried everything and found the real bottleneck isn’t how you set up the vector database—it’s data cleaning and query understanding. Here’s a breakdown of the pitfalls I ran into.
1. Don’t Rely on a Single OCR Library
Many assume that plugging in a large language model or an OCR library will just work. But when handling English error messages or file paths inside Chinese screenshots, pure-Chinese OCR engines fail completely. My solution: combine multiple models. Run a general-purpose OCR first, then pass low-confidence text blocks to a vision model for re-reading. Crucially, always preserve coordinate data—without it, you can’t highlight or locate text later.
2. Index Size Out of Control? Cut 60% First
Vectorizing everything is a trap—your index often ends up larger than the original images. Two key moves:
- Optimize chunking granularity: Don’t split by entire images; split by text blocks. Use regex, length thresholds, and positional rules to filter out timestamps, watermarks, and other useless UI text. This single step can eliminate up to 60% of unnecessary index entries.
- Use a two-tier retrieval architecture: Start with SQLite FTS5 for keyword-based coarse filtering, then run vector-based reranking only on the candidate set. This avoids the high latency of running ANN searches across the entire database.
3. What Does the User Mean by “That 502 Error”?
Query understanding sets the ceiling for retrieval. When a user types “that 502,” the system has no idea what they’re looking for. You must rewrite queries at the input layer: translate colloquial phrasing into forms likely to appear in screenshots, such as expanding “报错” into “error/failed/错误”. Also, parse time-related modifiers into separate filter conditions so they don’t pollute the vector semantics.
With this combined approach, 300+ days of screenshots respond in under a second with over 80% accuracy. Remember: the hardest engineering challenge isn’t the retrieval engine itself—it’s the upfront data cleaning and semantic understanding.
Source · V2EX Startup: Read original →