Slow Pandas Code Performance Review
Find and fix the slowest steps.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Review this pandas code for speed and memory. Code: [PASTE_CODE]. Data size and shape: [ROWS_COLUMNS_AND_MEMORY]. Current run time and the machine: [RUN_TIME_AND_MACHINE]. Python and pandas versions: [VERSIONS]. Return: 1. Measurement first: the exact code to profile it (df.info(memory_usage="deep"), %timeit or time.perf_counter, line-level timing) and what numbers to record before changing anything. 2. A table of the five most likely bottlenecks in my code: Line | Why it is slow (row-wise apply, iterrows, repeated concatenation in a loop, object dtype strings, chained indexing, unneeded copies) | Faster alternative | Expected gain (order of magnitude, labelled as an estimate) | Risk to correctness. 3. Memory fixes: downcast numerics, convert repeated strings to category, read only needed columns with usecols, read in chunks, use the pyarrow dtypes where supported. 4. The rewritten code for the top two fixes, with a test that the result equals the original using pd.testing.assert_frame_equal. 5. A rule for when pandas is the wrong tool (data larger than memory, many joins) and two alternatives to evaluate. Do not promise specific speedups; say what to measure.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.