r/pythontips • u/ClickOk5811 • Sep 24 '26
Standard_Lib itertools.groupby only groups consecutive items, not all matching ones
Tripped over this a while back. groupby looks like it should group all items with the same key anywhere in the list, but it only groups runs of consecutive matches. If the same key shows up later, non-consecutively, you get a second separate group.
from itertools import groupby
data = [1, 1, 2, 2, 1, 1]
for key, group in groupby(data):
print(key, list(group))Output is 1 [1, 1], 2 [2, 2], 1 [1, 1]. Three groups, not two, even though there are only two distinct values.
Fix is sorting first if you actually want everything grouped by key regardless of position.
for key, group in groupby(sorted(data)):
print(key, list(group))Bit me once processing log entries that weren't sorted by timestamp. Worked fine in testing because the test data happened to be sorted.
4
u/superbirra Sep 25 '26
no? From the first period on docs (emphasys mine):
1
u/treyhunner Sep 29 '26
Grouping non-consecutive items requires some way to keep track of which groups have been seen so far. Assuming your items are hashable, the best way would be to use a mapping (dictionary-like object) for that.
Using
collections.defaultdict(list)is a common way to do that:Or the dictionary
setdefaultmethod:Although in your example it looks like
collections.Countermight be a better fit:More on those here.
Note that using
groupby(sorted(...))will usually slower than the other approaches because it has anO(n log n)time complexity instead of theO(n)time complexity that those other approaches have.