Create intermediate Works with: Any AI

Logging and Checks Inside Data Scripts

Checklist Has example

Make scripts tell you when something is wrong.

The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.

Add logging and validation to my data script. Script purpose and steps: [SCRIPT_PURPOSE_AND_STEPS]. Inputs and outputs: [INPUTS_AND_OUTPUTS]. Where it runs: [WHERE_RUN].

Provide:
1. A logging setup using the standard logging module: levels, a format with timestamp and step, a file handler, and no personal data in logs.
2. Row-count logging at every transformation with a function that warns when rows change by more than a set percent.
3. Assertions and schema checks (required columns, dtypes, allowed values, key uniqueness) and a clear error message for each.
4. A summary log at the end: input files with checksums, output files with row counts, run time.
5. Exit codes and how a scheduler should react.
6. A small table of checks I should add, ranked by how often they catch real problems.
7. A test demonstrating that a bad input stops the script.
Write the code and explain each check in a comment.

Fill in before use

Optional: type your details and the prompt above updates. Nothing is saved or sent.

Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.

Logging: log the row count after each step.
import logging; logging.basicConfig(level=logging.INFO)
logging.info("Loaded %d rows", len(df))
Checks
assert df["id"].is_unique, "Duplicate ids"
assert df["amount"].ge(0).all(), "Negative amounts"
Better: raise ValueError with the offending rows (df.loc[df.amount < 0].head()).
Output: a short summary file with counts and the data version.
When a check fails: stop the script; do not continue silently.
Add a final reconciliation: total amount in vs out.
Example run output: INFO Loaded 12,480 rows; INFO After dedupe 12,310 rows; ERROR Negative amounts: 3 rows (ids 17, 44, 90). The script stops and prints the ids for review.

• script-logging