Database / pgvector basics Interview Questions
How do you bulk-load vectors efficiently into pgvector?
Inserting vectors one row at a time with individual INSERT statements is the slowest possible approach. For large datasets (thousands to millions of rows), bulk loading strategies dramatically reduce ingestion time.
-- Method 1: Multi-row INSERT (batch inserts) INSERT INTO documents (content, embedding) VALUES ('Document 1', '[0.1,0.2,0.3]'), ('Document 2', '[0.4,0.5,0.6]'), ('Document 3', '[0.7,0.8,0.9]'); -- Send batches of 100-1000 rows per INSERT statement -- Method 2: COPY command (fastest for very large datasets) COPY documents (content, embedding) FROM '/path/to/data.csv' WITH (FORMAT TEXT, DELIMITER '\t'); -- CSV format: text<TAB>[v1,v2,v3,...] -- Method 3: psycopg2 executemany or copy_from (Python) # cur.executemany( # "INSERT INTO documents (content, embedding) VALUES (%s, %s)", # [(text1, emb1), (text2, emb2), ...] # ) -- Method 4: Disable index during load, rebuild after -- (Much faster for large bulk loads) -- Step 1: Create table WITHOUT index CREATE TABLE documents ( id BIGSERIAL PRIMARY KEY, embedding VECTOR(1536) ); -- Step 2: Bulk load all data COPY documents FROM '/data/embeddings.csv'; -- Step 3: Increase memory and workers for faster index build SET maintenance_work_mem = '4GB'; SET max_parallel_maintenance_workers = 8; -- Step 4: Build the index AFTER loading all data CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops) WITH (m=16, ef_construction=64);
Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!
Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.
Invest now!!! Get Free equity stock (US, UK only)!
Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.
The Robinhood app makes it easy to trade stocks, crypto and more.
Webull! Receive free stock by signing up using the link: Webull signup.
More Related questions...
