Create intermediate Works with: Any AI

Hyperparameter Tuning Plan

Structured brief Has example

Tune sensibly without overfitting the validation set.

The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.

Plan hyperparameter tuning. Model type: [MODEL]. Data size and compute budget: [BUDGET]. Metric: [METRIC].

Explain which hyperparameters matter most for this model family and in what ranges to search (labelled as starting points to adapt). Compare grid, random, and Bayesian-style search and recommend one for my budget. Provide code skeleton with a pipeline so preprocessing is inside cross-validation, a log of trials, and a final evaluation on untouched test data. Add a stopping rule and a check for diminishing returns. Warn about tuning too long and about reporting the best validation score as the expected performance.

Add a trial log template and a rule for deciding when more tuning is less valuable than better features or more data. Provide a short example of reporting the final model with its tuning history, and guidance on setting random seeds and recording library versions.

Add a worked trial budget for an invented gradient-boosted model: a compute budget of 40 CPU-hours, a cross-validation run taking 6 minutes, therefore roughly 400 trials, split into 100 random-search trials over wide ranges, 200 narrower trials, and 100 reserved for repeats of the top configurations with different seeds. Label it an illustration. Add a decision table that shows when to stop (the best score has not improved by more than the seed-to-seed spread over the last fifty trials), a template for the final tuning report listing the search space, number of trials, selected configuration, validation score with spread, test score, and library versions, and a warning list of ways tuning leaks information, such as tuning on the test set or reusing a validation split many times. Finish with a compact summary line to record per trial (configuration, mean score, spread, runtime, seed) so the full history can be searched later without rereading notebooks.

Also add a short reminder to check the effect of random seeds by repeating the best configuration several times.

Fill in before use

Optional: type your details and the prompt above updates. Nothing is saved or sent.

Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.

Model: gradient boosting, 50,000 rows, budget 2 hours.
Starting ranges (adapt): learning rate 0.02-0.2 (log scale); depth 3-8; trees 200-1,000 with early stopping; subsample 0.6-1.0.
Method: random search, 60 trials, 5-fold grouped cross-validation, preprocessing inside the pipeline.
Log: trial id | params | mean score | std | time.
Final: refit on all training data and score once on the untouched test set.
Stop rule: stop when the best score has not improved by more than 0.002 for 15 trials.
Warning: the best validation score is optimistic; report the test score.

• ml
• structured-brief