search

Overview ¶

Package search is WIP experiment with semantic code search.

Transfer code files to vectors using transformers, saving vectors to database. Then allow to search for code places by meaning/prompt or code sample.

Current problems: - It can't be integrated in files data collection pipeline because it is too slow (5 seconds per 4000 symbols file); - SQLite vss (vector search) extension requires complex build pipeline and I won't use it. Alternative is using external vector DB (such as milvus, qdrant). - Search request prototype implemented though table full scan