pandas Performance Review: Loops, apply and Vectorisation
Find slow row-by-row code and safe replacements.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
[VERIFY] is an editorial flag, not a blank to fill. Wherever the prompt shows [VERIFY], check that item against a reliable source before you use it.
Review pandas code for performance. Code excerpt with sensitive details removed: [CODE_EXCERPT]. Data size: [DATA_SIZE]. Time taken now: [CURRENT_TIME]. Profile before optimising, and stop when the code is fast enough; micro-optimising small data is wasteful (for 1,000 rows it hardly matters, for 1,000,000 it does). Identify iterrows loops, apply with Python functions on large frames, repeated concatenation inside loops, chained indexing, unnecessary copies, and repeated reads of the same file. Table of patterns and replacements: iterrows to column arithmetic, apply with a lambda to a vectorised function or a map for lookups, repeated concatenation to collecting in a list then one concat, string operations with the string accessor, and conditional logic with where or select [VERIFY names]. Worked refactor in words: a loop over 200,000 rows calculating a discounted price is replaced by multiplying the price column by a factor with a conditional selection for the exceptions; timings as an example format (before 38 seconds, after 0.4 seconds, invented). Results must match exactly before the faster version replaces the slower one: check that the new column equals the old (zero differences) and the sum is identical. Provide a timing method with repeated runs and a before/after table, a table with Line, Problem, Fix and Check, and a rule to keep readability. Mark untested code or settings [VERIFY].
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.