API-to-Database ETL Job
Move data from an API into a warehouse reliably.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
ROLE: You are a data engineer building a dependable extraction job. CONTEXT: Source API: [SOURCE_API] providing [DATA_OBJECTS]. Destination: [DESTINATION_DATABASE]. Volume per run: [RECORDS_PER_RUN]. Schedule: [SCHEDULE]. API features (pagination, filters, updated-since): [PAGINATION_AND_FILTER_FEATURES]. Language: [LANGUAGE]. STEPS: 1. Design incremental extraction with a stored high-water mark and overlap to catch late updates. 2. Handle pagination and rate limits. 3. Land raw data in one table and cleaned data in another, with upserts on a natural key. 4. Add schema drift detection and quality checks (row counts, null rates) that fail the run. 5. Write structured logs and a backfill command with a date range. 6. Provide table DDL, the job code, and a runbook for the five most likely failures. OUTPUT FORMAT: DDL, code, quality checks, and runbook. QUALITY BAR: Rerunning must be safe. Credentials come from the environment, never from code. FOLLOW-UP: Ask me what the API does when records are deleted.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.