Wholesale Catalog Parser Pipeline

Challenge

A wholesale distributor needed daily snapshots of product availability and pricing from three supplier portals — each with different HTML structures, login requirements, and anti-automation measures. The procurement team was visiting each portal manually every morning to check stock before placing orders.

Solution

A Python Playwright-based multi-source parser pipeline: session management with credential rotation, human-like request intervals, structured output to PostgreSQL with a price and availability history table, daily export to Google Sheets for the procurement team, and Telegram alerts if any source fails to parse.

Results

Three supplier portals parsed automatically every morning before the procurement team starts their day. 1,200+ SKUs tracked with full availability and price history going back to deployment. The procurement team stopped visiting supplier portals manually — they now make ordering decisions from the Google Sheet and receive Telegram summaries of overnight changes.

Related service