Data automation & lead-gen systems
Data platforms and scraper operations for B2B lead generation, enrichment, deduplication, and reporting.
300+ tasks • 11 contributors
Challenge
Scrapers had to keep 7M+ records usable through target-site changes, proxy failures, migrations, and CRM syncs.
Outcome
Processed 7M+ records and 1M+ B2B leads for the DACH market.
My contribution
Coordinated scraper stability and data-quality fixes through AWS/server migrations, database updates and CRM enrichment. Managed client handoffs and reporting for the collection and synchronization workflows.
What shipped
- Pipelines that scraped 7M+ records and processed 1M+ B2B leads
- CRM enrichment and deduplication, synced to Pipedrive
- Looker Studio reporting
- Survived server migrations and repeated client handoffs
- Source-account and blocklist operations so the fleet survived site and access changes
Delivery team
Team mix: 5 backend, 2 frontend, 2 UI/UX, 1 DevOps, 1 QA.
Stack
- Python
- Django
- AWS
- nginx
- Docker
- SQL
- web scraping
- database sync
- CSV exports
- Pipedrive
- Looker Studio