Senior Data Engineer
Data Science
India · Remote
Firmable is the market-leading B2B sales intelligence platform in Asia pacific — and we're scaling that success globally at pace. Backed by leading investors and growing 2,000+ customers strong, we exist to give sales teams an unfair advantage: the deepest company and people data of any platform, enriched with real-time signals, served at the right moment by intelligent agents.
We're not building another search box. We're building the engine that tells a salesperson exactly who to call, why, and what to say — before they even ask.
The Role
This role sits at the heart of what makes Firmable's data valuable. As Senior Data Engineer - Sourcing, you'll own the harder extraction and ETL problems — the sites that fight back, the schemas that drift, the LLM pipelines that need to run as production systems, not parsing scripts.
You'll spend your time deep in Python, Airflow, and LLM extraction pipelines — architecting extractors that survive anti-bot defences and schema changes, building the eval scaffolding that holds LLM-based extraction accountable, and shipping ETL systems where rules, LLMs, and humans each do what they're best at. You'll treat LLMs as production infrastructure — versioned prompts, eval sets, traces, cost ceilings — not as a way to skip writing a parser.
This role is hands-on engineering with sharp judgement on extraction architecture. You decide when to write a parser, when to ask an LLM, and when to do both.
What You'll Own
Extractor Architecture & Web Collection at Scale
Architect resilient extractors for high-value, high-difficulty sources — sites with anti-bot defences, JS-heavy rendering, schema drift, or low-quality structure
Design LLM-based structured extraction as production systems — prompts with explicit rubrics, structured outputs, labelled eval sets, measured precision/recall
Make the rule-vs-LLM judgement call on every extractor: deterministic parsing where the structure allows, LLMs where semantic interpretation is genuinely needed — and justify the split
Lead proxy strategy and IP rotation architecture for high-volume, hard-to-reach sources
ETL Pipeline Engineering
Build production-grade ETL pipelines from extraction through normalisation, deduplication, and load into the warehouse
Design Airflow DAGs that recover gracefully, surface failures before they cascade, and don't wake people at 2am unnecessarily
Write performance-aware Python and SQL — extractors that run at scale across millions of records without falling over
Build the integration layer for partner APIs and third-party data sources
AI-Native Extraction Infrastructure
Ship agentic extraction pipelines: rule-based triage, LLM escalation for ambiguous content, structured output validation, retries on failure, and human review for the edges
Stand up the eval and observability layer for LLM extraction — prompt versioning, traces, drift detection when a vendor silently updates claude-sonnet-latest, cost ceilings with token math behind them
Build reusable skills (SKILL.md specs) for recurring extraction patterns — invocable by other engineers and agents
Choose the right model for the job — Haiku for cheap classification, Sonnet for nuanced extraction, frontier models for hard edge cases — and revise as model economics shift
Cross-Functional Partnership
Partner with data quality, product, and analytics on extraction schema, coverage, and accuracy standards
Translate sourcing requirements into extractor designs and ETL architecture
Be the technical voice for sourcing in cross-functional discussions on coverage and data quality
What We're Looking For
Must Haves
4+ years building production extraction, collection, or ETL pipelines in business-critical environments
Strong Python expertise — pandas, numpy, production-grade code, performance-aware. You write systems, not notebooks.
Advanced SQL — complex queries, performance optimisation, comfort across large datasets
Extensive Airflow (or equivalent) experience — end-to-end orchestration, dependency management, recovery patterns in production
Shipped real work with agentic IDEs — Claude Code, Cursor, or equivalent. Not "tried it" — built and merged real extraction systems with it.
Deep, demonstrable expertise integrating LLMs into extraction pipelines — explicit prompts with rubrics, structured outputs, eval sets, prompt versioning. You can show us the repos.
You operate LLMs as production systems — you've designed eval harnesses, run labelled eval sets, versioned prompts, logged traces, and debugged LLM extractions on precision/recall
Sharp judgement on rules vs. LLMs — you reach for a parser when the structure allows, and don't default to an LLM because it feels modern
Strong knowledge of web extraction at scale — anti-bot defences, proxy strategy, JS rendering, schema drift handling
A product mindset — you understand that extraction quality directly impacts customer value
Highly Valued
Experience with cloud data platforms — Snowflake, Databricks, Redshift, or RDS
Hands-on AWS experience for pipeline deployment (Lambda, S3, EC2, Glue)
Understanding of data warehousing concepts and how extraction feeds downstream systems
Experience building reusable agents, skills, or tool-calling pipelines that other engineers or agents invoke
Exposure to vector databases and embeddings for matching and deduplication
Knowledge of data privacy and compliance considerations
How We Build
AI-Native, Not AI-Assisted
Firmable is built on an AI-native engineering philosophy — and we mean it literally. AI is not a productivity tool bolted onto traditional workflows. AI is the workflow. Every engineer at Firmable operates with fully agentic development, evals, traces, and AI-powered review pipelines as their default mode of working.
This means:
Agentic development: extractors, pipelines, and infrastructure are designed, scaffolded, and iterated with AI agents doing the heavy lifting — you direct, review, and elevate
Skills over scripts: recurring workflows are packaged as versioned SKILL.md specs that any teammate or agent can load and run
Evals as a default, not an afterthought: every LLM extraction ships with a labelled eval set, measured precision/recall, and a prompt version you can roll back
Traces and observability from day one: every LLM call is logged with prompt version, model, cost, latency, and decision — retrofitting this later is not the plan
Continuous AI feedback loops: model drift, prompt regression, and cost ceilings are monitored the same way pipeline health is
If you're not already working this way, this role will require a rapid and genuine mindset shift. We're not looking for people who are open to AI-native work — we're looking for people who already live it.
The Operating Environment
Firmable runs lean and ships fast — intentionally small teams, no layers, minimal process, and a weekly release cadence moving toward daily. Teams own their stack end to end: you design it, you build it, you ship it, you run it.
This is a startup-to-scaleup environment and it comes with real expectations. There are no fixed hours. The pace is high, the team is always building, and when something matters it gets done. In return, you get genuine ownership, a seat at the table on every major architecture decision, and the opportunity to build something that doesn't exist anywhere else in the market.
Why This Role
Own the extraction systems behind one of the fastest-growing B2B intelligence platforms in APAC — every record starts in a pipeline you architect
Greenfield AI-native scaffolding — the eval harnesses, skills library, and agentic extraction pipelines are largely unbuilt; you'll shape them
Work at the frontier — LLM-based extraction at production scale, agentic crawling, drift detection on vendor models, and rule-vs-LLM orchestration
Small team, massive leverage — your extractors reach every Firmable customer, every day
Competitive base + meaningful equity — we balance strong compensation with a share in the upside we're building toward
Firmable is an equal opportunity employer. We believe diverse teams build better products.
