More Itertools

Python’s itertools library is a gem - you can compose elegant solutions for a variety of problems with the functions it provides. In more-itertools we collect additional building blocks, recipes, and routines for working with Python iterables.

Grouping	chunked, ichunked, chunked_even, sliced, constrained_batches, distribute, divide, split_at, split_before, split_after, split_into, split_when, bucket, unzip, batched, grouper, partition transpose
Lookahead and lookback	spy, peekable, seekable
Windowing	windowed, substrings, substrings_indexes, stagger, windowed_complete, pairwise, triplewise, sliding_window, subslices
Augmenting	count_cycle, intersperse, padded, repeat_each, mark_ends, repeat_last, adjacent, groupby_transform, pad_none, ncycles
Combining	collapse, sort_together, interleave, interleave_longest, interleave_evenly, zip_offset, zip_equal, zip_broadcast, dotproduct, convolve, flatten, roundrobin, prepend, value_chain
Summarizing	ilen, unique_to_each, sample, consecutive_groups, run_length, map_reduce, exactly_n, is_sorted, all_equal, all_unique, minmax, first_true, quantify, iequals
Selecting	islice_extended, first, last, one, only, strictly_n, strip, lstrip, rstrip, filter_except, map_except, nth_or_last, unique_in_window, before_and_after, nth, take, tail, unique_everseen, unique_justseen, duplicates_everseen, duplicates_justseen, longest_common_prefix
Combinatorics	distinct_permutations, distinct_combinations, circular_shifts, partitions, set_partitions, product_index, combination_index, permutation_index, gray_product, powerset, random_product, random_permutation, random_combination, random_combination_with_replacement, nth_product, nth_permutation, nth_combination
Wrapping	always_iterable, always_reversible, countable, consumer, with_iter, iter_except
Others	locate, rlocate, replace, numeric_range, side_effect, iterate, difference, make_decorator, SequenceView, time_limited, map_if, iter_index, consume, tabulate, repeatfunc, polynomial_from_roots, sieve factor matmul

Getting started

To get started, install the library with pip:

pip install more-itertools

The recipes from the itertools docs are included in the top-level package:

>>> from more_itertools import flatten
>>> iterable = [(0, 1), (2, 3)]
>>> list(flatten(iterable))
[0, 1, 2, 3]

Several new recipes are available as well:

>>> from more_itertools import chunked
>>> iterable = [0, 1, 2, 3, 4, 5, 6, 7, 8]
>>> list(chunked(iterable, 3))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

>>> from more_itertools import spy
>>> iterable = (x * x for x in range(1, 6))
>>> head, iterable = spy(iterable, n=3)
>>> list(head)
[1, 4, 9]
>>> list(iterable)
[1, 4, 9, 16, 25]

For the full listing of functions, see the API documentation.

Links elsewhere

Blog posts about more-itertools:

• Yo, I heard you like decorators
• Tour of Python Itertools (Alternate)
• Real-World Python More Itertools

Development

more-itertools is maintained by @erikrose and @bbayles, with help from many others. If you have a problem or suggestion, please file a bug or pull request in this repository. Thanks for contributing!

Version History

The version history can be found in documentation.

Contents

API Reference

More routines for operating on iterables, beyond itertools

Grouping

These tools yield groups of items from a source iterable.

---

New itertools

more_itertools.chunked(iterable, n, strict=False)

Break iterable into lists of length n:

>>> list(chunked([1, 2, 3, 4, 5, 6], 3))
[[1, 2, 3], [4, 5, 6]]

By the default, the last yielded list will have fewer than n elements if the length of iterable is not divisible by n:

>>> list(chunked([1, 2, 3, 4, 5, 6, 7, 8], 3))
[[1, 2, 3], [4, 5, 6], [7, 8]]

To use a fill-in value instead, see the grouper() recipe.

If the length of iterable is not divisible by n and strict is True, then ValueError will be raised before the last list is yielded.

more_itertools.ichunked(iterable, n)

Break iterable into sub-iterables with n elements each. ichunked() is like chunked(), but it yields iterables instead of lists.

If the sub-iterables are read in order, the elements of iterable won’t be stored in memory. If they are read out of order, itertools.tee() is used to cache elements as necessary.

>>> from itertools import count
>>> all_chunks = ichunked(count(), 4)
>>> c_1, c_2, c_3 = next(all_chunks), next(all_chunks), next(all_chunks)
>>> list(c_2)  # c_1's elements have been cached; c_3's haven't been
[4, 5, 6, 7]
>>> list(c_1)
[0, 1, 2, 3]
>>> list(c_3)
[8, 9, 10, 11]

more_itertools.chunked_even(iterable, n)

Break iterable into lists of approximately length n. Items are distributed such the lengths of the lists differ by at most 1 item.

>>> iterable = [1, 2, 3, 4, 5, 6, 7]
>>> n = 3
>>> list(chunked_even(iterable, n))  # List lengths: 3, 2, 2
[[1, 2, 3], [4, 5], [6, 7]]
>>> list(chunked(iterable, n))  # List lengths: 3, 3, 1
[[1, 2, 3], [4, 5, 6], [7]]

more_itertools.sliced(seq, n, strict=False)

Yield slices of length n from the sequence seq.

>>> list(sliced((1, 2, 3, 4, 5, 6), 3))
[(1, 2, 3), (4, 5, 6)]

By the default, the last yielded slice will have fewer than n elements if the length of seq is not divisible by n:

>>> list(sliced((1, 2, 3, 4, 5, 6, 7, 8), 3))
[(1, 2, 3), (4, 5, 6), (7, 8)]

If the length of seq is not divisible by n and strict is True, then ValueError will be raised before the last slice is yielded.

This function will only work for iterables that support slicing. For non-sliceable iterables, see chunked().

more_itertools.constrained_batches(iterable, max_size, max_count=None, get_len=len, strict=True)

Yield batches of items from iterable with a combined size limited by max_size.

>>> iterable = [b'12345', b'123', b'12345678', b'1', b'1', b'12', b'1']
>>> list(constrained_batches(iterable, 10))
[(b'12345', b'123'), (b'12345678', b'1', b'1'), (b'12', b'1')]

If a max_count is supplied, the number of items per batch is also limited:

>>> iterable = [b'12345', b'123', b'12345678', b'1', b'1', b'12', b'1']
>>> list(constrained_batches(iterable, 10, max_count = 2))
[(b'12345', b'123'), (b'12345678', b'1'), (b'1', b'12'), (b'1',)]

If a get_len function is supplied, use that instead of len() to determine item size.

If strict is True, raise ValueError if any single item is bigger than max_size. Otherwise, allow single items to exceed max_size.

more_itertools.distribute(n, iterable)

Distribute the items from iterable among n smaller iterables.

>>> group_1, group_2 = distribute(2, [1, 2, 3, 4, 5, 6])
>>> list(group_1)
[1, 3, 5]
>>> list(group_2)
[2, 4, 6]

If the length of iterable is not evenly divisible by n, then the length of the returned iterables will not be identical:

>>> children = distribute(3, [1, 2, 3, 4, 5, 6, 7])
>>> [list(c) for c in children]
[[1, 4, 7], [2, 5], [3, 6]]

If the length of iterable is smaller than n, then the last returned iterables will be empty:

>>> children = distribute(5, [1, 2, 3])
>>> [list(c) for c in children]
[[1], [2], [3], [], []]

This function uses itertools.tee() and may require significant storage. If you need the order items in the smaller iterables to match the original iterable, see divide().

more_itertools.divide(n, iterable)

Divide the elements from iterable into n parts, maintaining order.

>>> group_1, group_2 = divide(2, [1, 2, 3, 4, 5, 6])
>>> list(group_1)
[1, 2, 3]
>>> list(group_2)
[4, 5, 6]

If the length of iterable is not evenly divisible by n, then the length of the returned iterables will not be identical:

>>> children = divide(3, [1, 2, 3, 4, 5, 6, 7])
>>> [list(c) for c in children]
[[1, 2, 3], [4, 5], [6, 7]]

If the length of the iterable is smaller than n, then the last returned iterables will be empty:

>>> children = divide(5, [1, 2, 3])
>>> [list(c) for c in children]
[[1], [2], [3], [], []]

This function will exhaust the iterable before returning and may require significant storage. If order is not important, see distribute(), which does not first pull the iterable into memory.

more_itertools.split_at(iterable, pred, maxsplit=-1, keep_separator=False)

Yield lists of items from iterable, where each list is delimited by an item where callable pred returns True.

>>> list(split_at('abcdcba', lambda x: x == 'b'))
[['a'], ['c', 'd', 'c'], ['a']]

>>> list(split_at(range(10), lambda n: n % 2 == 1))
[[0], [2], [4], [6], [8], []]

At most maxsplit splits are done. If maxsplit is not specified or -1, then there is no limit on the number of splits:

>>> list(split_at(range(10), lambda n: n % 2 == 1, maxsplit=2))
[[0], [2], [4, 5, 6, 7, 8, 9]]

By default, the delimiting items are not included in the output. To include them, set keep_separator to True.

>>> list(split_at('abcdcba', lambda x: x == 'b', keep_separator=True))
[['a'], ['b'], ['c', 'd', 'c'], ['b'], ['a']]

more_itertools.split_before(iterable, pred, maxsplit=-1)

Yield lists of items from iterable, where each list ends just before an item for which callable pred returns True:

>>> list(split_before('OneTwo', lambda s: s.isupper()))
[['O', 'n', 'e'], ['T', 'w', 'o']]

>>> list(split_before(range(10), lambda n: n % 3 == 0))
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

At most maxsplit splits are done. If maxsplit is not specified or -1, then there is no limit on the number of splits:

>>> list(split_before(range(10), lambda n: n % 3 == 0, maxsplit=2))
[[0, 1, 2], [3, 4, 5], [6, 7, 8, 9]]

more_itertools.split_after(iterable, pred, maxsplit=-1)

Yield lists of items from iterable, where each list ends with an item where callable pred returns True:

>>> list(split_after('one1two2', lambda s: s.isdigit()))
[['o', 'n', 'e', '1'], ['t', 'w', 'o', '2']]

>>> list(split_after(range(10), lambda n: n % 3 == 0))
[[0], [1, 2, 3], [4, 5, 6], [7, 8, 9]]

At most maxsplit splits are done. If maxsplit is not specified or -1, then there is no limit on the number of splits:

>>> list(split_after(range(10), lambda n: n % 3 == 0, maxsplit=2))
[[0], [1, 2, 3], [4, 5, 6, 7, 8, 9]]

more_itertools.split_into(iterable, sizes)

Yield a list of sequential items from iterable of length ‘n’ for each integer ‘n’ in sizes.

>>> list(split_into([1,2,3,4,5,6], [1,2,3]))
[[1], [2, 3], [4, 5, 6]]

If the sum of sizes is smaller than the length of iterable, then the remaining items of iterable will not be returned.

>>> list(split_into([1,2,3,4,5,6], [2,3]))
[[1, 2], [3, 4, 5]]

If the sum of sizes is larger than the length of iterable, fewer items will be returned in the iteration that overruns iterable and further lists will be empty:

>>> list(split_into([1,2,3,4], [1,2,3,4]))
[[1], [2, 3], [4], []]

When a None object is encountered in sizes, the returned list will contain items up to the end of iterable the same way that itertools.slice does:

>>> list(split_into([1,2,3,4,5,6,7,8,9,0], [2,3,None]))
[[1, 2], [3, 4, 5], [6, 7, 8, 9, 0]]

split_into() can be useful for grouping a series of items where the sizes of the groups are not uniform. An example would be where in a row from a table, multiple columns represent elements of the same feature (e.g. a point represented by x,y,z) but, the format is not the same for all columns.

more_itertools.split_when(iterable, pred, maxsplit=-1)

Split iterable into pieces based on the output of pred. pred should be a function that takes successive pairs of items and returns True if the iterable should be split in between them.

For example, to find runs of increasing numbers, split the iterable when element i is larger than element i + 1:

>>> list(split_when([1, 2, 3, 3, 2, 5, 2, 4, 2], lambda x, y: x > y))
[[1, 2, 3, 3], [2, 5], [2, 4], [2]]

At most maxsplit splits are done. If maxsplit is not specified or -1, then there is no limit on the number of splits:

>>> list(split_when([1, 2, 3, 3, 2, 5, 2, 4, 2],
...                 lambda x, y: x > y, maxsplit=2))
[[1, 2, 3, 3], [2, 5], [2, 4, 2]]

more_itertools.bucket(iterable, key, validator=None)

Wrap iterable and return an object that buckets it iterable into child iterables based on a key function.

>>> iterable = ['a1', 'b1', 'c1', 'a2', 'b2', 'c2', 'b3']
>>> s = bucket(iterable, key=lambda x: x[0])  # Bucket by 1st character
>>> sorted(list(s))  # Get the keys
['a', 'b', 'c']
>>> a_iterable = s['a']
>>> next(a_iterable)
'a1'
>>> next(a_iterable)
'a2'
>>> list(s['b'])
['b1', 'b2', 'b3']

The original iterable will be advanced and its items will be cached until they are used by the child iterables. This may require significant storage.

By default, attempting to select a bucket to which no items belong will exhaust the iterable and cache all values. If you specify a validator function, selected buckets will instead be checked against it.

>>> from itertools import count
>>> it = count(1, 2)  # Infinite sequence of odd numbers
>>> key = lambda x: x % 10  # Bucket by last digit
>>> validator = lambda x: x in {1, 3, 5, 7, 9}  # Odd digits only
>>> s = bucket(it, key=key, validator=validator)
>>> 2 in s
False
>>> list(s[2])
[]

more_itertools.unzip(iterable)

The inverse of zip(), this function disaggregates the elements of the zipped iterable.

The i-th iterable contains the i-th element from each element of the zipped iterable. The first element is used to determine the length of the remaining elements.

>>> iterable = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> letters, numbers = unzip(iterable)
>>> list(letters)
['a', 'b', 'c', 'd']
>>> list(numbers)
[1, 2, 3, 4]

This is similar to using zip(*iterable), but it avoids reading iterable into memory. Note, however, that this function uses itertools.tee() and thus may require significant storage.

---

Itertools recipes

more_itertools.batched(iterable, n)

Batch data into lists of length n. The last batch may be shorter.

>>> list(batched('ABCDEFG', 3))
[['A', 'B', 'C'], ['D', 'E', 'F'], ['G']]

This recipe is from the itertools docs. This library also provides chunked(), which has a different implementation.

more_itertools.grouper(iterable, n, incomplete='fill', fillvalue=None)

Group elements from iterable into fixed-length groups of length n.

>>> list(grouper('ABCDEF', 3))
[('A', 'B', 'C'), ('D', 'E', 'F')]

The keyword arguments incomplete and fillvalue control what happens for iterables whose length is not a multiple of n.

When incomplete is ‘fill’, the last group will contain instances of fillvalue.

>>> list(grouper('ABCDEFG', 3, incomplete='fill', fillvalue='x'))
[('A', 'B', 'C'), ('D', 'E', 'F'), ('G', 'x', 'x')]

When incomplete is ‘ignore’, the last group will not be emitted.

>>> list(grouper('ABCDEFG', 3, incomplete='ignore', fillvalue='x'))
[('A', 'B', 'C'), ('D', 'E', 'F')]

When incomplete is ‘strict’, a subclass of ValueError will be raised.

>>> it = grouper('ABCDEFG', 3, incomplete='strict')
>>> list(it)  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
UnequalIterablesError

more_itertools.partition(pred, iterable)

Returns a 2-tuple of iterables derived from the input iterable. The first yields the items that have pred(item) == False. The second yields the items that have pred(item) == True.

>>> is_odd = lambda x: x % 2 != 0
>>> iterable = range(10)
>>> even_items, odd_items = partition(is_odd, iterable)
>>> list(even_items), list(odd_items)
([0, 2, 4, 6, 8], [1, 3, 5, 7, 9])

If pred is None, bool() is used.

>>> iterable = [0, 1, False, True, '', ' ']
>>> false_items, true_items = partition(None, iterable)
>>> list(false_items), list(true_items)
([0, False, ''], [1, True, ' '])

more_itertools.transpose(it)

Swap the rows and columns of the input.

>>> list(transpose([(1, 2, 3), (11, 22, 33)]))
[(1, 11), (2, 22), (3, 33)]

The caller should ensure that the dimensions of the input are compatible.

Lookahead and lookback

These tools peek at an iterable’s values without advancing it.

---

New itertools

more_itertools.spy(iterable, n=1)

Return a 2-tuple with a list containing the first n elements of iterable, and an iterator with the same items as iterable. This allows you to “look ahead” at the items in the iterable without advancing it.

There is one item in the list by default:

>>> iterable = 'abcdefg'
>>> head, iterable = spy(iterable)
>>> head
['a']
>>> list(iterable)
['a', 'b', 'c', 'd', 'e', 'f', 'g']

You may use unpacking to retrieve items instead of lists:

>>> (head,), iterable = spy('abcdefg')
>>> head
'a'
>>> (first, second), iterable = spy('abcdefg', 2)
>>> first
'a'
>>> second
'b'

The number of items requested can be larger than the number of items in the iterable:

>>> iterable = [1, 2, 3, 4, 5]
>>> head, iterable = spy(iterable, 10)
>>> head
[1, 2, 3, 4, 5]
>>> list(iterable)
[1, 2, 3, 4, 5]

class more_itertools.peekable(iterable)

Wrap an iterator to allow lookahead and prepending elements.

Call peek() on the result to get the value that will be returned by next(). This won’t advance the iterator:

>>> p = peekable(['a', 'b'])
>>> p.peek()
'a'
>>> next(p)
'a'

Pass peek() a default value to return that instead of raising StopIteration when the iterator is exhausted.

>>> p = peekable([])
>>> p.peek('hi')
'hi'

peekables also offer a prepend() method, which “inserts” items at the head of the iterable:

>>> p = peekable([1, 2, 3])
>>> p.prepend(10, 11, 12)
>>> next(p)
10
>>> p.peek()
11
>>> list(p)
[11, 12, 1, 2, 3]

peekables can be indexed. Index 0 is the item that will be returned by next(), index 1 is the item after that, and so on: The values up to the given index will be cached.

>>> p = peekable(['a', 'b', 'c', 'd'])
>>> p[0]
'a'
>>> p[1]
'b'
>>> next(p)
'a'

Negative indexes are supported, but be aware that they will cache the remaining items in the source iterator, which may require significant storage.

To check whether a peekable is exhausted, check its truth value:

>>> p = peekable(['a', 'b'])
>>> if p:  # peekable has items
...     list(p)
['a', 'b']
>>> if not p:  # peekable is exhausted
...     list(p)
[]

class more_itertools.seekable(iterable, maxlen=None)

Wrap an iterator to allow for seeking backward and forward. This progressively caches the items in the source iterable so they can be re-visited.

Call seek() with an index to seek to that position in the source iterable.

To “reset” an iterator, seek to 0:

>>> from itertools import count
>>> it = seekable((str(n) for n in count()))
>>> next(it), next(it), next(it)
('0', '1', '2')
>>> it.seek(0)
>>> next(it), next(it), next(it)
('0', '1', '2')
>>> next(it)
'3'

You can also seek forward:

>>> it = seekable((str(n) for n in range(20)))
>>> it.seek(10)
>>> next(it)
'10'
>>> it.seek(20)  # Seeking past the end of the source isn't a problem
>>> list(it)
[]
>>> it.seek(0)  # Resetting works even after hitting the end
>>> next(it), next(it), next(it)
('0', '1', '2')

Call peek() to look ahead one item without advancing the iterator:

>>> it = seekable('1234')
>>> it.peek()
'1'
>>> list(it)
['1', '2', '3', '4']
>>> it.peek(default='empty')
'empty'

Before the iterator is at its end, calling bool() on it will return True. After it will return False:

>>> it = seekable('5678')
>>> bool(it)
True
>>> list(it)
['5', '6', '7', '8']
>>> bool(it)
False

You may view the contents of the cache with the elements() method. That returns a SequenceView, a view that updates automatically:

>>> it = seekable((str(n) for n in range(10)))
>>> next(it), next(it), next(it)
('0', '1', '2')
>>> elements = it.elements()
>>> elements
SequenceView(['0', '1', '2'])
>>> next(it)
'3'
>>> elements
SequenceView(['0', '1', '2', '3'])

By default, the cache grows as the source iterable progresses, so beware of wrapping very large or infinite iterables. Supply maxlen to limit the size of the cache (this of course limits how far back you can seek).

>>> from itertools import count
>>> it = seekable((str(n) for n in count()), maxlen=2)
>>> next(it), next(it), next(it), next(it)
('0', '1', '2', '3')
>>> list(it.elements())
['2', '3']
>>> it.seek(0)
>>> next(it), next(it), next(it), next(it)
('2', '3', '4', '5')
>>> next(it)
'6'

Windowing

These tools yield windows of items from an iterable.

---

New itertools

more_itertools.windowed(seq, n, fillvalue=None, step=1)

Return a sliding window of width n over the given iterable.

>>> all_windows = windowed([1, 2, 3, 4, 5], 3)
>>> list(all_windows)
[(1, 2, 3), (2, 3, 4), (3, 4, 5)]

When the window is larger than the iterable, fillvalue is used in place of missing values:

>>> list(windowed([1, 2, 3], 4))
[(1, 2, 3, None)]

Each window will advance in increments of step:

>>> list(windowed([1, 2, 3, 4, 5, 6], 3, fillvalue='!', step=2))
[(1, 2, 3), (3, 4, 5), (5, 6, '!')]

To slide into the iterable’s items, use chain() to add filler items to the left:

>>> iterable = [1, 2, 3, 4]
>>> n = 3
>>> padding = [None] * (n - 1)
>>> list(windowed(chain(padding, iterable), 3))
[(None, None, 1), (None, 1, 2), (1, 2, 3), (2, 3, 4)]

more_itertools.substrings(iterable)

Yield all of the substrings of iterable.

>>> [''.join(s) for s in substrings('more')]
['m', 'o', 'r', 'e', 'mo', 'or', 're', 'mor', 'ore', 'more']

Note that non-string iterables can also be subdivided.

>>> list(substrings([0, 1, 2]))
[(0,), (1,), (2,), (0, 1), (1, 2), (0, 1, 2)]

more_itertools.substrings_indexes(seq, reverse=False)

Yield all substrings and their positions in seq

The items yielded will be a tuple of the form (substr, i, j), where substr == seq[i:j].

This function only works for iterables that support slicing, such as str objects.

>>> for item in substrings_indexes('more'):
...    print(item)
('m', 0, 1)
('o', 1, 2)
('r', 2, 3)
('e', 3, 4)
('mo', 0, 2)
('or', 1, 3)
('re', 2, 4)
('mor', 0, 3)
('ore', 1, 4)
('more', 0, 4)

Set reverse to True to yield the same items in the opposite order.

more_itertools.stagger(iterable, offsets=(-1, 0, 1), longest=False, fillvalue=None)

Yield tuples whose elements are offset from iterable. The amount by which the i-th item in each tuple is offset is given by the i-th item in offsets.

>>> list(stagger([0, 1, 2, 3]))
[(None, 0, 1), (0, 1, 2), (1, 2, 3)]
>>> list(stagger(range(8), offsets=(0, 2, 4)))
[(0, 2, 4), (1, 3, 5), (2, 4, 6), (3, 5, 7)]

By default, the sequence will end when the final element of a tuple is the last item in the iterable. To continue until the first element of a tuple is the last item in the iterable, set longest to True:

>>> list(stagger([0, 1, 2, 3], longest=True))
[(None, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, None), (3, None, None)]

By default, None will be used to replace offsets beyond the end of the sequence. Specify fillvalue to use some other value.

more_itertools.windowed_complete(iterable, n)

Yield (beginning, middle, end) tuples, where:

• Each middle has n items from iterable
• Each beginning has the items before the ones in middle
• Each end has the items after the ones in middle

>>> iterable = range(7)
>>> n = 3
>>> for beginning, middle, end in windowed_complete(iterable, n):
...     print(beginning, middle, end)
() (0, 1, 2) (3, 4, 5, 6)
(0,) (1, 2, 3) (4, 5, 6)
(0, 1) (2, 3, 4) (5, 6)
(0, 1, 2) (3, 4, 5) (6,)
(0, 1, 2, 3) (4, 5, 6) ()

Note that n must be at least 0 and most equal to the length of iterable.

This function will exhaust the iterable and may require significant storage.

---

Itertools recipes

more_itertools.pairwise(iterable)

Returns an iterator of paired items, overlapping, from the original

>>> take(4, pairwise(count()))
[(0, 1), (1, 2), (2, 3), (3, 4)]

On Python 3.10 and above, this is an alias for itertools.pairwise().

more_itertools.triplewise(iterable)

Return overlapping triplets from iterable.

>>> list(triplewise('ABCDE'))
[('A', 'B', 'C'), ('B', 'C', 'D'), ('C', 'D', 'E')]

more_itertools.sliding_window(iterable, n)

Return a sliding window of width n over iterable.

>>> list(sliding_window(range(6), 4))
[(0, 1, 2, 3), (1, 2, 3, 4), (2, 3, 4, 5)]

If iterable has fewer than n items, then nothing is yielded:

>>> list(sliding_window(range(3), 4))
[]

For a variant with more features, see windowed().

more_itertools.subslices(iterable)

Return all contiguous non-empty subslices of iterable.

>>> list(subslices('ABC'))
[['A'], ['A', 'B'], ['A', 'B', 'C'], ['B'], ['B', 'C'], ['C']]

This is similar to substrings(), but emits items in a different order.

Augmenting

These tools yield items from an iterable, plus additional data.

---

New itertools

more_itertools.count_cycle(iterable, n=None)

Cycle through the items from iterable up to n times, yielding the number of completed cycles along with each item. If n is omitted the process repeats indefinitely.

>>> list(count_cycle('AB', 3))
[(0, 'A'), (0, 'B'), (1, 'A'), (1, 'B'), (2, 'A'), (2, 'B')]

more_itertools.intersperse(e, iterable, n=1)

Intersperse filler element e among the items in iterable, leaving n items between each filler element.

>>> list(intersperse('!', [1, 2, 3, 4, 5]))
[1, '!', 2, '!', 3, '!', 4, '!', 5]

>>> list(intersperse(None, [1, 2, 3, 4, 5], n=2))
[1, 2, None, 3, 4, None, 5]

more_itertools.padded(iterable, fillvalue=None, n=None, next_multiple=False)

Yield the elements from iterable, followed by fillvalue, such that at least n items are emitted.

>>> list(padded([1, 2, 3], '?', 5))
[1, 2, 3, '?', '?']

If next_multiple is True, fillvalue will be emitted until the number of items emitted is a multiple of n:

>>> list(padded([1, 2, 3, 4], n=3, next_multiple=True))
[1, 2, 3, 4, None, None]

If n is None, fillvalue will be emitted indefinitely.

more_itertools.mark_ends(iterable)

Yield 3-tuples of the form (is_first, is_last, item).

>>> list(mark_ends('ABC'))
[(True, False, 'A'), (False, False, 'B'), (False, True, 'C')]

Use this when looping over an iterable to take special action on its first and/or last items:

>>> iterable = ['Header', 100, 200, 'Footer']
>>> total = 0
>>> for is_first, is_last, item in mark_ends(iterable):
...     if is_first:
...         continue  # Skip the header
...     if is_last:
...         continue  # Skip the footer
...     total += item
>>> print(total)
300

more_itertools.repeat_each(iterable, n=2)

Repeat each element in iterable n times.

>>> list(repeat_each('ABC', 3))
['A', 'A', 'A', 'B', 'B', 'B', 'C', 'C', 'C']

more_itertools.repeat_last(iterable, default=None)

After the iterable is exhausted, keep yielding its last element.

>>> list(islice(repeat_last(range(3)), 5))
[0, 1, 2, 2, 2]

If the iterable is empty, yield default forever:

>>> list(islice(repeat_last(range(0), 42), 5))
[42, 42, 42, 42, 42]

more_itertools.adjacent(predicate, iterable, distance=1)

Return an iterable over (bool, item) tuples where the item is drawn from iterable and the bool indicates whether that item satisfies the predicate or is adjacent to an item that does.

For example, to find whether items are adjacent to a 3:

>>> list(adjacent(lambda x: x == 3, range(6)))
[(False, 0), (False, 1), (True, 2), (True, 3), (True, 4), (False, 5)]

Set distance to change what counts as adjacent. For example, to find whether items are two places away from a 3:

>>> list(adjacent(lambda x: x == 3, range(6), distance=2))
[(False, 0), (True, 1), (True, 2), (True, 3), (True, 4), (True, 5)]

This is useful for contextualizing the results of a search function. For example, a code comparison tool might want to identify lines that have changed, but also surrounding lines to give the viewer of the diff context.

The predicate function will only be called once for each item in the iterable.

See also groupby_transform(), which can be used with this function to group ranges of items with the same bool value.

more_itertools.groupby_transform(iterable, keyfunc=None, valuefunc=None, reducefunc=None)

An extension of itertools.groupby() that can apply transformations to the grouped data.

• keyfunc is a function computing a key value for each item in iterable
• valuefunc is a function that transforms the individual items from iterable after grouping
• reducefunc is a function that transforms each group of items

>>> iterable = 'aAAbBBcCC'
>>> keyfunc = lambda k: k.upper()
>>> valuefunc = lambda v: v.lower()
>>> reducefunc = lambda g: ''.join(g)
>>> list(groupby_transform(iterable, keyfunc, valuefunc, reducefunc))
[('A', 'aaa'), ('B', 'bbb'), ('C', 'ccc')]

Each optional argument defaults to an identity function if not specified.

groupby_transform() is useful when grouping elements of an iterable using a separate iterable as the key. To do this, zip() the iterables and pass a keyfunc that extracts the first element and a valuefunc that extracts the second element:

>>> from operator import itemgetter
>>> keys = [0, 0, 1, 1, 1, 2, 2, 2, 3]
>>> values = 'abcdefghi'
>>> iterable = zip(keys, values)
>>> grouper = groupby_transform(iterable, itemgetter(0), itemgetter(1))
>>> [(k, ''.join(g)) for k, g in grouper]
[(0, 'ab'), (1, 'cde'), (2, 'fgh'), (3, 'i')]

Note that the order of items in the iterable is significant. Only adjacent items are grouped together, so if you don’t want any duplicate groups, you should sort the iterable by the key function.

---

Itertools recipes

more_itertools.padnone()

more_itertools.pad_none(iterable)

Returns the sequence of elements and then returns None indefinitely.

>>> take(5, pad_none(range(3)))
[0, 1, 2, None, None]

Useful for emulating the behavior of the built-in map() function.

See also padded().

more_itertools.ncycles(iterable, n)

Returns the sequence elements n times

>>> list(ncycles(["a", "b"], 3))
['a', 'b', 'a', 'b', 'a', 'b']

Combining

These tools combine multiple iterables.

---

New itertools

more_itertools.collapse(iterable, base_type=None, levels=None)

Flatten an iterable with multiple levels of nesting (e.g., a list of lists of tuples) into non-iterable types.

>>> iterable = [(1, 2), ([3, 4], [[5], [6]])]
>>> list(collapse(iterable))
[1, 2, 3, 4, 5, 6]

Binary and text strings are not considered iterable and will not be collapsed.

To avoid collapsing other types, specify base_type:

>>> iterable = ['ab', ('cd', 'ef'), ['gh', 'ij']]
>>> list(collapse(iterable, base_type=tuple))
['ab', ('cd', 'ef'), 'gh', 'ij']

Specify levels to stop flattening after a certain level:

>>> iterable = [('a', ['b']), ('c', ['d'])]
>>> list(collapse(iterable))  # Fully flattened
['a', 'b', 'c', 'd']
>>> list(collapse(iterable, levels=1))  # Only one level flattened
['a', ['b'], 'c', ['d']]

more_itertools.interleave(*iterables)

Return a new iterable yielding from each iterable in turn, until the shortest is exhausted.

>>> list(interleave([1, 2, 3], [4, 5], [6, 7, 8]))
[1, 4, 6, 2, 5, 7]

For a version that doesn’t terminate after the shortest iterable is exhausted, see interleave_longest().

more_itertools.interleave_longest(*iterables)

Return a new iterable yielding from each iterable in turn, skipping any that are exhausted.

>>> list(interleave_longest([1, 2, 3], [4, 5], [6, 7, 8]))
[1, 4, 6, 2, 5, 7, 3, 8]

This function produces the same output as roundrobin(), but may perform better for some inputs (in particular when the number of iterables is large).

more_itertools.interleave_evenly(iterables, lengths=None)

Interleave multiple iterables so that their elements are evenly distributed throughout the output sequence.

>>> iterables = [1, 2, 3, 4, 5], ['a', 'b']
>>> list(interleave_evenly(iterables))
[1, 2, 'a', 3, 4, 'b', 5]

>>> iterables = [[1, 2, 3], [4, 5], [6, 7, 8]]
>>> list(interleave_evenly(iterables))
[1, 6, 4, 2, 7, 3, 8, 5]

This function requires iterables of known length. Iterables without __len__() can be used by manually specifying lengths with lengths:

>>> from itertools import combinations, repeat
>>> iterables = [combinations(range(4), 2), ['a', 'b', 'c']]
>>> lengths = [4 * (4 - 1) // 2, 3]
>>> list(interleave_evenly(iterables, lengths=lengths))
[(0, 1), (0, 2), 'a', (0, 3), (1, 2), 'b', (1, 3), (2, 3), 'c']

Based on Bresenham’s algorithm.

more_itertools.sort_together(iterables, key_list=(0, ), key=None, reverse=False)

Return the input iterables sorted together, with key_list as the priority for sorting. All iterables are trimmed to the length of the shortest one.

This can be used like the sorting function in a spreadsheet. If each iterable represents a column of data, the key list determines which columns are used for sorting.

By default, all iterables are sorted using the 0-th iterable:

>>> iterables = [(4, 3, 2, 1), ('a', 'b', 'c', 'd')]
>>> sort_together(iterables)
[(1, 2, 3, 4), ('d', 'c', 'b', 'a')]

Set a different key list to sort according to another iterable. Specifying multiple keys dictates how ties are broken:

>>> iterables = [(3, 1, 2), (0, 1, 0), ('c', 'b', 'a')]
>>> sort_together(iterables, key_list=(1, 2))
[(2, 3, 1), (0, 0, 1), ('a', 'c', 'b')]

To sort by a function of the elements of the iterable, pass a key function. Its arguments are the elements of the iterables corresponding to the key list:

>>> names = ('a', 'b', 'c')
>>> lengths = (1, 2, 3)
>>> widths = (5, 2, 1)
>>> def area(length, width):
...     return length * width
>>> sort_together([names, lengths, widths], key_list=(1, 2), key=area)
[('c', 'b', 'a'), (3, 2, 1), (1, 2, 5)]

Set reverse to True to sort in descending order.

>>> sort_together([(1, 2, 3), ('c', 'b', 'a')], reverse=True)
[(3, 2, 1), ('a', 'b', 'c')]

more_itertools.value_chain(*args)

Yield all arguments passed to the function in the same order in which they were passed. If an argument itself is iterable then iterate over its values.

>>> list(value_chain(1, 2, 3, [4, 5, 6]))
[1, 2, 3, 4, 5, 6]

Binary and text strings are not considered iterable and are emitted as-is:

>>> list(value_chain('12', '34', ['56', '78']))
['12', '34', '56', '78']

Multiple levels of nesting are not flattened.

more_itertools.zip_offset(*iterables, offsets, longest=False, fillvalue=None)

zip the input iterables together, but offset the i-th iterable by the i-th item in offsets.

>>> list(zip_offset('0123', 'abcdef', offsets=(0, 1)))
[('0', 'b'), ('1', 'c'), ('2', 'd'), ('3', 'e')]

This can be used as a lightweight alternative to SciPy or pandas to analyze data sets in which some series have a lead or lag relationship.

By default, the sequence will end when the shortest iterable is exhausted. To continue until the longest iterable is exhausted, set longest to True.

>>> list(zip_offset('0123', 'abcdef', offsets=(0, 1), longest=True))
[('0', 'b'), ('1', 'c'), ('2', 'd'), ('3', 'e'), (None, 'f')]

By default, None will be used to replace offsets beyond the end of the sequence. Specify fillvalue to use some other value.

more_itertools.zip_equal(*iterables)

zip the input iterables together, but raise UnequalIterablesError if they aren’t all the same length.

>>> it_1 = range(3)
>>> it_2 = iter('abc')
>>> list(zip_equal(it_1, it_2))
[(0, 'a'), (1, 'b'), (2, 'c')]

>>> it_1 = range(3)
>>> it_2 = iter('abcd')
>>> list(zip_equal(it_1, it_2)) # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
more_itertools.more.UnequalIterablesError: Iterables have different
lengths

more_itertools.zip_broadcast(*objects, scalar_types=(str, bytes), strict=False)

A version of zip() that “broadcasts” any scalar (i.e., non-iterable) items into output tuples.

>>> iterable_1 = [1, 2, 3]
>>> iterable_2 = ['a', 'b', 'c']
>>> scalar = '_'
>>> list(zip_broadcast(iterable_1, iterable_2, scalar))
[(1, 'a', '_'), (2, 'b', '_'), (3, 'c', '_')]

The scalar_types keyword argument determines what types are considered scalar. It is set to (str, bytes) by default. Set it to None to treat strings and byte strings as iterable:

>>> list(zip_broadcast('abc', 0, 'xyz', scalar_types=None))
[('a', 0, 'x'), ('b', 0, 'y'), ('c', 0, 'z')]

If the strict keyword argument is True, then UnequalIterablesError will be raised if any of the iterables have different lengths.

---

Itertools recipes

more_itertools.dotproduct(vec1, vec2)

Returns the dot product of the two iterables.

>>> dotproduct([10, 10], [20, 20])
400

more_itertools.convolve(signal, kernel)

Convolve the iterable signal with the iterable kernel.

>>> signal = (1, 2, 3, 4, 5)
>>> kernel = [3, 2, 1]
>>> list(convolve(signal, kernel))
[3, 8, 14, 20, 26, 14, 5]

Note: the input arguments are not interchangeable, as the kernel is immediately consumed and stored.

more_itertools.flatten(listOfLists)

Return an iterator flattening one level of nesting in a list of lists.

>>> list(flatten([[0, 1], [2, 3]]))
[0, 1, 2, 3]

See also collapse(), which can flatten multiple levels of nesting.

more_itertools.roundrobin(*iterables)

Yields an item from each iterable, alternating between them.

>>> list(roundrobin('ABC', 'D', 'EF'))
['A', 'D', 'E', 'B', 'F', 'C']

This function produces the same output as interleave_longest(), but may perform better for some inputs (in particular when the number of iterables is small).

more_itertools.prepend(value, iterator)

Yield value, followed by the elements in iterator.

>>> value = '0'
>>> iterator = ['1', '2', '3']
>>> list(prepend(value, iterator))
['0', '1', '2', '3']

To prepend multiple values, see itertools.chain() or value_chain().

Summarizing

These tools return summarized or aggregated data from an iterable.

---

New itertools

more_itertools.ilen(iterable)

Return the number of items in iterable.

>>> ilen(x for x in range(1000000) if x % 3 == 0)
333334

This consumes the iterable, so handle with care.

more_itertools.unique_to_each(*iterables)

Return the elements from each of the input iterables that aren’t in the other input iterables.

For example, suppose you have a set of packages, each with a set of dependencies:

{'pkg_1': {'A', 'B'}, 'pkg_2': {'B', 'C'}, 'pkg_3': {'B', 'D'}}

If you remove one package, which dependencies can also be removed?

If pkg_1 is removed, then A is no longer necessary - it is not associated with pkg_2 or pkg_3. Similarly, C is only needed for pkg_2, and D is only needed for pkg_3:

>>> unique_to_each({'A', 'B'}, {'B', 'C'}, {'B', 'D'})
[['A'], ['C'], ['D']]

If there are duplicates in one input iterable that aren’t in the others they will be duplicated in the output. Input order is preserved:

>>> unique_to_each("mississippi", "missouri")
[['p', 'p'], ['o', 'u', 'r']]

It is assumed that the elements of each iterable are hashable.

more_itertools.sample(iterable, k=1, weights=None)

Return a k-length list of elements chosen (without replacement) from the iterable. Like random.sample(), but works on iterables of unknown length.

>>> iterable = range(100)
>>> sample(iterable, 5)  # doctest: +SKIP
[81, 60, 96, 16, 4]

An iterable with weights may also be given:

>>> iterable = range(100)
>>> weights = (i * i + 1 for i in range(100))
>>> sampled = sample(iterable, 5, weights=weights)  # doctest: +SKIP
[79, 67, 74, 66, 78]

The algorithm can also be used to generate weighted random permutations. The relative weight of each item determines the probability that it appears late in the permutation.

>>> data = "abcdefgh"
>>> weights = range(1, len(data) + 1)
>>> sample(data, k=len(data), weights=weights)  # doctest: +SKIP
['c', 'a', 'b', 'e', 'g', 'd', 'h', 'f']

more_itertools.consecutive_groups(iterable, ordering=lambda x: x)

Yield groups of consecutive items using itertools.groupby(). The ordering function determines whether two items are adjacent by returning their position.

By default, the ordering function is the identity function. This is suitable for finding runs of numbers:

>>> iterable = [1, 10, 11, 12, 20, 30, 31, 32, 33, 40]
>>> for group in consecutive_groups(iterable):
...     print(list(group))
[1]
[10, 11, 12]
[20]
[30, 31, 32, 33]
[40]

For finding runs of adjacent letters, try using the index() method of a string of letters:

>>> from string import ascii_lowercase
>>> iterable = 'abcdfgilmnop'
>>> ordering = ascii_lowercase.index
>>> for group in consecutive_groups(iterable, ordering):
...     print(list(group))
['a', 'b', 'c', 'd']
['f', 'g']
['i']
['l', 'm', 'n', 'o', 'p']

Each group of consecutive items is an iterator that shares it source with iterable. When an an output group is advanced, the previous group is no longer available unless its elements are copied (e.g., into a list).

>>> iterable = [1, 2, 11, 12, 21, 22]
>>> saved_groups = []
>>> for group in consecutive_groups(iterable):
...     saved_groups.append(list(group))  # Copy group elements
>>> saved_groups
[[1, 2], [11, 12], [21, 22]]

class more_itertools.run_length

run_length.encode() compresses an iterable with run-length encoding. It yields groups of repeated items with the count of how many times they were repeated:

>>> uncompressed = 'abbcccdddd'
>>> list(run_length.encode(uncompressed))
[('a', 1), ('b', 2), ('c', 3), ('d', 4)]

run_length.decode() decompresses an iterable that was previously compressed with run-length encoding. It yields the items of the decompressed iterable:

>>> compressed = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> list(run_length.decode(compressed))
['a', 'b', 'b', 'c', 'c', 'c', 'd', 'd', 'd', 'd']

more_itertools.map_reduce(iterable, keyfunc, valuefunc=None, reducefunc=None)

Return a dictionary that maps the items in iterable to categories defined by keyfunc, transforms them with valuefunc, and then summarizes them by category with reducefunc.

valuefunc defaults to the identity function if it is unspecified. If reducefunc is unspecified, no summarization takes place:

>>> keyfunc = lambda x: x.upper()
>>> result = map_reduce('abbccc', keyfunc)
>>> sorted(result.items())
[('A', ['a']), ('B', ['b', 'b']), ('C', ['c', 'c', 'c'])]

Specifying valuefunc transforms the categorized items:

>>> keyfunc = lambda x: x.upper()
>>> valuefunc = lambda x: 1
>>> result = map_reduce('abbccc', keyfunc, valuefunc)
>>> sorted(result.items())
[('A', [1]), ('B', [1, 1]), ('C', [1, 1, 1])]

Specifying reducefunc summarizes the categorized items:

>>> keyfunc = lambda x: x.upper()
>>> valuefunc = lambda x: 1
>>> reducefunc = sum
>>> result = map_reduce('abbccc', keyfunc, valuefunc, reducefunc)
>>> sorted(result.items())
[('A', 1), ('B', 2), ('C', 3)]

You may want to filter the input iterable before applying the map/reduce procedure:

>>> all_items = range(30)
>>> items = [x for x in all_items if 10 <= x <= 20]  # Filter
>>> keyfunc = lambda x: x % 2  # Evens map to 0; odds to 1
>>> categories = map_reduce(items, keyfunc=keyfunc)
>>> sorted(categories.items())
[(0, [10, 12, 14, 16, 18, 20]), (1, [11, 13, 15, 17, 19])]
>>> summaries = map_reduce(items, keyfunc=keyfunc, reducefunc=sum)
>>> sorted(summaries.items())
[(0, 90), (1, 75)]

Note that all items in the iterable are gathered into a list before the summarization step, which may require significant storage.

The returned object is a collections.defaultdict with the default_factory set to None, such that it behaves like a normal dictionary.

more_itertools.exactly_n(iterable, n, predicate=bool)

Return True if exactly n items in the iterable are True according to the predicate function.

>>> exactly_n([True, True, False], 2)
True
>>> exactly_n([True, True, False], 1)
False
>>> exactly_n([0, 1, 2, 3, 4, 5], 3, lambda x: x < 3)
True

The iterable will be advanced until n + 1 truthy items are encountered, so avoid calling it on infinite iterables.

more_itertools.is_sorted(iterable, key=None, reverse=False, strict=False)

Returns True if the items of iterable are in sorted order, and False otherwise. key and reverse have the same meaning that they do in the built-in sorted() function.

>>> is_sorted(['1', '2', '3', '4', '5'], key=int)
True
>>> is_sorted([5, 4, 3, 1, 2], reverse=True)
False

If strict, tests for strict sorting, that is, returns False if equal elements are found:

>>> is_sorted([1, 2, 2])
True
>>> is_sorted([1, 2, 2], strict=True)
False

The function returns False after encountering the first out-of-order item. If there are no out-of-order items, the iterable is exhausted.

more_itertools.all_unique(iterable, key=None)

Returns True if all the elements of iterable are unique (no two elements are equal).

>>> all_unique('ABCB')
False

If a key function is specified, it will be used to make comparisons.

>>> all_unique('ABCb')
True
>>> all_unique('ABCb', str.lower)
False

The function returns as soon as the first non-unique element is encountered. Iterables with a mix of hashable and unhashable items can be used, but the function will be slower for unhashable items.

more_itertools.minmax(iterable, *[, key, default])

more_itertools.minmax(arg1, arg2, *args[, key])

Returns both the smallest and largest items in an iterable or the largest of two or more arguments.

>>> minmax([3, 1, 5])
(1, 5)

>>> minmax(4, 2, 6)
(2, 6)

If a key function is provided, it will be used to transform the input items for comparison.

>>> minmax([5, 30], key=str)  # '30' sorts before '5'
(30, 5)

If a default value is provided, it will be returned if there are no input items.

>>> minmax([], default=(0, 0))
(0, 0)

Otherwise ValueError is raised.

This function is based on the recipe by Raymond Hettinger and takes care to minimize the number of comparisons performed.

more_itertools.iequals(*iterables)

Return True if all given iterables are equal to each other, which means that they contain the same elements in the same order.

The function is useful for comparing iterables of different data types or iterables that do not support equality checks.

>>> iequals("abc", ['a', 'b', 'c'], ('a', 'b', 'c'), iter("abc"))
True

>>> iequals("abc", "acb")
False

Not to be confused with all_equals(), which checks whether all elements of iterable are equal to each other.

---

Itertools recipes

more_itertools.all_equal(iterable)

Returns True if all the elements are equal to each other.

>>> all_equal('aaaa')
True
>>> all_equal('aaab')
False

more_itertools.first_true(iterable, default=None, pred=None)

Returns the first true value in the iterable.

If no true value is found, returns default

If pred is not None, returns the first item for which pred(item) == True .

>>> first_true(range(10))
1
>>> first_true(range(10), pred=lambda x: x > 5)
6
>>> first_true(range(10), default='missing', pred=lambda x: x > 9)
'missing'

more_itertools.quantify(iterable, pred=bool)

Return the how many times the predicate is true.

>>> quantify([True, False, True])
2

Selecting

These tools yield certain items from an iterable.

---

New itertools

class more_itertools.islice_extended(iterable, stop)

class more_itertools.islice_extended(iterable, start, stop[, step])

An extension of itertools.islice() that supports negative values for stop, start, and step.

>>> iterable = iter('abcdefgh')
>>> list(islice_extended(iterable, -4, -1))
['e', 'f', 'g']

Slices with negative values require some caching of iterable, but this function takes care to minimize the amount of memory required.

For example, you can use a negative step with an infinite iterator:

>>> from itertools import count
>>> list(islice_extended(count(), 110, 99, -2))
[110, 108, 106, 104, 102, 100]

You can also use slice notation directly:

>>> iterable = map(str, count())
>>> it = islice_extended(iterable)[10:20:2]
>>> list(it)
['10', '12', '14', '16', '18']

more_itertools.first(iterable[, default])

Return the first item of iterable, or default if iterable is empty.

>>> first([0, 1, 2, 3])
0
>>> first([], 'some default')
'some default'

If default is not provided and there are no items in the iterable, raise ValueError.

first() is useful when you have a generator of expensive-to-retrieve values and want any arbitrary one. It is marginally shorter than next(iter(iterable), default).

more_itertools.last(iterable[, default])

Return the last item of iterable, or default if iterable is empty.

>>> last([0, 1, 2, 3])
3
>>> last([], 'some default')
'some default'

If default is not provided and there are no items in the iterable, raise ValueError.

more_itertools.one(iterable, too_short=ValueError, too_long=ValueError)

Return the first item from iterable, which is expected to contain only that item. Raise an exception if iterable is empty or has more than one item.

one() is useful for ensuring that an iterable contains only one item. For example, it can be used to retrieve the result of a database query that is expected to return a single row.

If iterable is empty, ValueError will be raised. You may specify a different exception with the too_short keyword:

>>> it = []
>>> one(it)  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
ValueError: too many items in iterable (expected 1)'
>>> too_short = IndexError('too few items')
>>> one(it, too_short=too_short)  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
IndexError: too few items

Similarly, if iterable contains more than one item, ValueError will be raised. You may specify a different exception with the too_long keyword:

>>> it = ['too', 'many']
>>> one(it)  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
ValueError: Expected exactly one item in iterable, but got 'too',
'many', and perhaps more.
>>> too_long = RuntimeError
>>> one(it, too_long=too_long)  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
RuntimeError

Note that one() attempts to advance iterable twice to ensure there is only one item. See spy() or peekable() to check iterable contents less destructively.

more_itertools.only(iterable, default=None, too_long=ValueError)

If iterable has only one item, return it. If it has zero items, return default. If it has more than one item, raise the exception given by too_long, which is ValueError by default.

>>> only([], default='missing')
'missing'
>>> only([1])
1
>>> only([1, 2])  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
ValueError: Expected exactly one item in iterable, but got 1, 2,
 and perhaps more.'
>>> only([1, 2], too_long=TypeError)  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
TypeError

Note that only() attempts to advance iterable twice to ensure there is only one item. See spy() or peekable() to check iterable contents less destructively.

more_itertools.strictly_n(iterable, too_short=None, too_long=None)

Validate that iterable has exactly n items and return them if it does. If it has fewer than n items, call function too_short with those items. If it has more than n items, call function too_long with the first n + 1 items.

>>> iterable = ['a', 'b', 'c', 'd']
>>> n = 4
>>> list(strictly_n(iterable, n))
['a', 'b', 'c', 'd']

By default, too_short and too_long are functions that raise ValueError.

>>> list(strictly_n('ab', 3))  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
ValueError: too few items in iterable (got 2)

>>> list(strictly_n('abc', 2))  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
ValueError: too many items in iterable (got at least 3)

You can instead supply functions that do something else. too_short will be called with the number of items in iterable. too_long will be called with n + 1.

>>> def too_short(item_count):
...     raise RuntimeError
>>> it = strictly_n('abcd', 6, too_short=too_short)
>>> list(it)  # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
RuntimeError

>>> def too_long(item_count):
...     print('The boss is going to hear about this')
>>> it = strictly_n('abcdef', 4, too_long=too_long)
>>> list(it)
The boss is going to hear about this
['a', 'b', 'c', 'd']

more_itertools.strip(iterable, pred)

Yield the items from iterable, but strip any from the beginning and end for which pred returns True.

For example, to remove a set of items from both ends of an iterable:

>>> iterable = (None, False, None, 1, 2, None, 3, False, None)
>>> pred = lambda x: x in {None, False, ''}
>>> list(strip(iterable, pred))
[1, 2, None, 3]

This function is analogous to str.strip().

more_itertools.lstrip(iterable, pred)

Yield the items from iterable, but strip any from the beginning for which pred returns True.

For example, to remove a set of items from the start of an iterable:

>>> iterable = (None, False, None, 1, 2, None, 3, False, None)
>>> pred = lambda x: x in {None, False, ''}
>>> list(lstrip(iterable, pred))
[1, 2, None, 3, False, None]

This function is analogous to to str.lstrip(), and is essentially an wrapper for itertools.dropwhile().

more_itertools.rstrip(iterable, pred)

Yield the items from iterable, but strip any from the end for which pred returns True.

For example, to remove a set of items from the end of an iterable:

>>> iterable = (None, False, None, 1, 2, None, 3, False, None)
>>> pred = lambda x: x in {None, False, ''}
>>> list(rstrip(iterable, pred))
[None, False, None, 1, 2, None, 3]

This function is analogous to str.rstrip().

more_itertools.filter_except(validator, iterable, *exceptions)

Yield the items from iterable for which the validator function does not raise one of the specified exceptions.

validator is called for each item in iterable. It should be a function that accepts one argument and raises an exception if that item is not valid.

>>> iterable = ['1', '2', 'three', '4', None]
>>> list(filter_except(int, iterable, ValueError, TypeError))
['1', '2', '4']

If an exception other than one given by exceptions is raised by validator, it is raised like normal.

more_itertools.map_except(function, iterable, *exceptions)

Transform each item from iterable with function and yield the result, unless function raises one of the specified exceptions.

function is called to transform each item in iterable. It should accept one argument.

>>> iterable = ['1', '2', 'three', '4', None]
>>> list(map_except(int, iterable, ValueError, TypeError))
[1, 2, 4]

If an exception other than one given by exceptions is raised by function, it is raised like normal.

more_itertools.nth_or_last(iterable, n[, default])

Return the nth or the last item of iterable, or default if iterable is empty.

>>> nth_or_last([0, 1, 2, 3], 2)
2
>>> nth_or_last([0, 1], 2)
1
>>> nth_or_last([], 0, 'some default')
'some default'

If default is not provided and there are no items in the iterable, raise ValueError.

more_itertools.unique_in_window(iterable, n, key=None)

Yield the items from iterable that haven’t been seen recently. n is the size of the lookback window.

>>> iterable = [0, 1, 0, 2, 3, 0]
>>> n = 3
>>> list(unique_in_window(iterable, n))
[0, 1, 2, 3, 0]

The key function, if provided, will be used to determine uniqueness:

>>> list(unique_in_window('abAcda', 3, key=lambda x: x.lower()))
['a', 'b', 'c', 'd', 'a']

The items in iterable must be hashable.

more_itertools.duplicates_everseen(iterable, key=None)

Yield duplicate elements after their first appearance.

>>> list(duplicates_everseen('mississippi'))
['s', 'i', 's', 's', 'i', 'p', 'i']
>>> list(duplicates_everseen('AaaBbbCccAaa', str.lower))
['a', 'a', 'b', 'b', 'c', 'c', 'A', 'a', 'a']

This function is analagous to unique_everseen() and is subject to the same performance considerations.

more_itertools.duplicates_justseen(iterable, key=None)

Yields serially-duplicate elements after their first appearance.

>>> list(duplicates_justseen('mississippi'))
['s', 's', 'p']
>>> list(duplicates_justseen('AaaBbbCccAaa', str.lower))
['a', 'a', 'b', 'b', 'c', 'c', 'a', 'a']

This function is analagous to unique_justseen().

more_itertools.longest_common_prefix(iterables)

Yield elements of the longest common prefix amongst given iterables.

>>> ''.join(longest_common_prefix(['abcd', 'abc', 'abf']))
'ab'

---

Itertools recipes

more_itertools.nth(iterable, n, default=None)

Returns the nth item or a default value.

>>> l = range(10)
>>> nth(l, 3)
3
>>> nth(l, 20, "zebra")
'zebra'

more_itertools.before_and_after(predicate, it)

A variant of takewhile() that allows complete access to the remainder of the iterator.

>>> it = iter('ABCdEfGhI')
>>> all_upper, remainder = before_and_after(str.isupper, it)
>>> ''.join(all_upper)
'ABC'
>>> ''.join(remainder) # takewhile() would lose the 'd'
'dEfGhI'

Note that the first iterator must be fully consumed before the second iterator can generate valid results.

more_itertools.take(n, iterable)

Return first n items of the iterable as a list.

>>> take(3, range(10))
[0, 1, 2]

If there are fewer than n items in the iterable, all of them are returned.

>>> take(10, range(3))
[0, 1, 2]

more_itertools.tail(n, iterable)

Return an iterator over the last n items of iterable.

>>> t = tail(3, 'ABCDEFG')
>>> list(t)
['E', 'F', 'G']

more_itertools.unique_everseen(iterable, key=None)

Yield unique elements, preserving order.

>>> list(unique_everseen('AAAABBBCCDAABBB'))
['A', 'B', 'C', 'D']
>>> list(unique_everseen('ABBCcAD', str.lower))
['A', 'B', 'C', 'D']

Sequences with a mix of hashable and unhashable items can be used. The function will be slower (i.e., O(n^2)) for unhashable items.

Remember that list objects are unhashable - you can use the key parameter to transform the list to a tuple (which is hashable) to avoid a slowdown.

>>> iterable = ([1, 2], [2, 3], [1, 2])
>>> list(unique_everseen(iterable))  # Slow
[[1, 2], [2, 3]]
>>> list(unique_everseen(iterable, key=tuple))  # Faster
[[1, 2], [2, 3]]

Similary, you may want to convert unhashable set objects with key=frozenset. For dict objects, key=lambda x: frozenset(x.items()) can be used.

more_itertools.unique_justseen(iterable, key=None)

Yields elements in order, ignoring serial duplicates

>>> list(unique_justseen('AAAABBBCCDAABBB'))
['A', 'B', 'C', 'D', 'A', 'B']
>>> list(unique_justseen('ABBCcAD', str.lower))
['A', 'B', 'C', 'A', 'D']

Combinatorics

These tools yield combinatorial arrangements of items from iterables.

---

New itertools

more_itertools.distinct_permutations(iterable, r=None)

Yield successive distinct permutations of the elements in iterable.

>>> sorted(distinct_permutations([1, 0, 1]))
[(0, 1, 1), (1, 0, 1), (1, 1, 0)]

Equivalent to set(permutations(iterable)), except duplicates are not generated and thrown away. For larger input sequences this is much more efficient.

Duplicate permutations arise when there are duplicated elements in the input iterable. The number of items returned is n! / (x_1! * x_2! * … * x_n!), where n is the total number of items input, and each x_i is the count of a distinct item in the input sequence.

If r is given, only the r-length permutations are yielded.

>>> sorted(distinct_permutations([1, 0, 1], r=2))
[(0, 1), (1, 0), (1, 1)]
>>> sorted(distinct_permutations(range(3), r=2))
[(0, 1), (0, 2), (1, 0), (1, 2), (2, 0), (2, 1)]

more_itertools.distinct_combinations(iterable, r)

Yield the distinct combinations of r items taken from iterable.

>>> list(distinct_combinations([0, 0, 1], 2))
[(0, 0), (0, 1)]

Equivalent to set(combinations(iterable)), except duplicates are not generated and thrown away. For larger input sequences this is much more efficient.

more_itertools.circular_shifts(iterable)

Return a list of circular shifts of iterable.

>>> circular_shifts(range(4))
[(0, 1, 2, 3), (1, 2, 3, 0), (2, 3, 0, 1), (3, 0, 1, 2)]

more_itertools.partitions(iterable)

Yield all possible order-preserving partitions of iterable.

>>> iterable = 'abc'
>>> for part in partitions(iterable):
...     print([''.join(p) for p in part])
['abc']
['a', 'bc']
['ab', 'c']
['a', 'b', 'c']

This is unrelated to partition().

more_itertools.set_partitions(iterable, k=None)

Yield the set partitions of iterable into k parts. Set partitions are not order-preserving.

>>> iterable = 'abc'
>>> for part in set_partitions(iterable, 2):
...     print([''.join(p) for p in part])
['a', 'bc']
['ab', 'c']
['b', 'ac']

If k is not given, every set partition is generated.

>>> iterable = 'abc'
>>> for part in set_partitions(iterable):
...     print([''.join(p) for p in part])
['abc']
['a', 'bc']
['ab', 'c']
['b', 'ac']
['a', 'b', 'c']

more_itertools.product_index(element, *args)

Equivalent to list(product(*args)).index(element)

The products of args can be ordered lexicographically. product_index() computes the first index of element without computing the previous products.

>>> product_index([8, 2], range(10), range(5))
42

ValueError will be raised if the given element isn’t in the product of args.

more_itertools.combination_index(element, iterable)

Equivalent to list(combinations(iterable, r)).index(element)

The subsequences of iterable that are of length r can be ordered lexicographically. combination_index() computes the index of the first element, without computing the previous combinations.

>>> combination_index('adf', 'abcdefg')
10

ValueError will be raised if the given element isn’t one of the combinations of iterable.

more_itertools.permutation_index(element, iterable)

Equivalent to list(permutations(iterable, r)).index(element)`

The subsequences of iterable that are of length r where order is important can be ordered lexicographically. permutation_index() computes the index of the first element directly, without computing the previous permutations.

>>> permutation_index([1, 3, 2], range(5))
19

ValueError will be raised if the given element isn’t one of the permutations of iterable.

more_itertools.gray_product(*iterables)

Like itertools.product(), but return tuples in an order such that only one element in the generated tuple changes from one iteration to the next.

>>> list(gray_product('AB','CD'))
[('A', 'C'), ('B', 'C'), ('B', 'D'), ('A', 'D')]

This function consumes all of the input iterables before producing output. If any of the input iterables have fewer than two items, ValueError is raised.

For information on the algorithm, see this section of Donald Knuth’s The Art of Computer Programming.

---

Itertools recipes

more_itertools.powerset(iterable)

Yields all possible subsets of the iterable.

>>> list(powerset([1, 2, 3]))
[(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)]

powerset() will operate on iterables that aren’t set instances, so repeated elements in the input will produce repeated elements in the output. Use unique_everseen() on the input to avoid generating duplicates:

>>> seq = [1, 1, 0]
>>> list(powerset(seq))
[(), (1,), (1,), (0,), (1, 1), (1, 0), (1, 0), (1, 1, 0)]
>>> from more_itertools import unique_everseen
>>> list(powerset(unique_everseen(seq)))
[(), (1,), (0,), (1, 0)]

more_itertools.random_product(*args, repeat=1)

Draw an item at random from each of the input iterables.

>>> random_product('abc', range(4), 'XYZ')  # doctest:+SKIP
('c', 3, 'Z')

If repeat is provided as a keyword argument, that many items will be drawn from each iterable.

>>> random_product('abcd', range(4), repeat=2)  # doctest:+SKIP
('a', 2, 'd', 3)

This equivalent to taking a random selection from itertools.product(*args, **kwarg).

more_itertools.random_permutation(iterable, r=None)

Return a random r length permutation of the elements in iterable.

If r is not specified or is None, then r defaults to the length of iterable.

>>> random_permutation(range(5))  # doctest:+SKIP
(3, 4, 0, 1, 2)

This equivalent to taking a random selection from itertools.permutations(iterable, r).

more_itertools.random_combination(iterable, r)

Return a random r length subsequence of the elements in iterable.

>>> random_combination(range(5), 3)  # doctest:+SKIP
(2, 3, 4)

This equivalent to taking a random selection from itertools.combinations(iterable, r).

more_itertools.random_combination_with_replacement(iterable, r)

Return a random r length subsequence of elements in iterable, allowing individual elements to be repeated.

>>> random_combination_with_replacement(range(3), 5) # doctest:+SKIP
(0, 0, 1, 2, 2)

This equivalent to taking a random selection from itertools.combinations_with_replacement(iterable, r).

more_itertools.nth_product(index, *args)

Equivalent to list(product(*args))[index].

The products of args can be ordered lexicographically. nth_product() computes the product at sort position index without computing the previous products.

>>> nth_product(8, range(2), range(2), range(2), range(2))
(1, 0, 0, 0)

IndexError will be raised if the given index is invalid.

more_itertools.nth_permutation(iterable, r, index)

Equivalent to list(permutations(iterable, r))[index]`

The subsequences of iterable that are of length r where order is important can be ordered lexicographically. nth_permutation() computes the subsequence at sort position index directly, without computing the previous subsequences.

>>> nth_permutation('ghijk', 2, 5)
('h', 'i')

ValueError will be raised If r is negative or greater than the length of iterable. IndexError will be raised if the given index is invalid.

more_itertools.nth_combination(iterable, r, index)

Equivalent to list(combinations(iterable, r))[index].

The subsequences of iterable that are of length r can be ordered lexicographically. nth_combination() computes the subsequence at sort position index directly, without computing the previous subsequences.

>>> nth_combination(range(5), 3, 5)
(0, 3, 4)

ValueError will be raised If r is negative or greater than the length of iterable. IndexError will be raised if the given index is invalid.

Wrapping

These tools provide wrappers to smooth working with objects that produce or consume iterables.

---

New itertools

more_itertools.always_iterable(obj, base_type=(<class 'str'>, <class 'bytes'>))

If obj is iterable, return an iterator over its items:

>>> obj = (1, 2, 3)
>>> list(always_iterable(obj))
[1, 2, 3]

If obj is not iterable, return a one-item iterable containing obj:

>>> obj = 1
>>> list(always_iterable(obj))
[1]

If obj is None, return an empty iterable:

>>> obj = None
>>> list(always_iterable(None))
[]

By default, binary and text strings are not considered iterable:

>>> obj = 'foo'
>>> list(always_iterable(obj))
['foo']

If base_type is set, objects for which isinstance(obj, base_type) returns True won’t be considered iterable.

>>> obj = {'a': 1}
>>> list(always_iterable(obj))  # Iterate over the dict's keys
['a']
>>> list(always_iterable(obj, base_type=dict))  # Treat dicts as a unit
[{'a': 1}]

Set base_type to None to avoid any special handling and treat objects Python considers iterable as iterable:

>>> obj = 'foo'
>>> list(always_iterable(obj, base_type=None))
['f', 'o', 'o']

more_itertools.always_reversible(iterable)

An extension of reversed() that supports all iterables, not just those which implement the Reversible or Sequence protocols.

>>> print(*always_reversible(x for x in range(3)))
2 1 0

If the iterable is already reversible, this function returns the result of reversed(). If the iterable is not reversible, this function will cache the remaining items in the iterable and yield them in reverse order, which may require significant storage.

more_itertools.countable(iterable)

Wrap iterable and keep a count of how many items have been consumed.

The items_seen attribute starts at 0 and increments as the iterable is consumed:

>>> iterable = map(str, range(10))
>>> it = countable(iterable)
>>> it.items_seen
0
>>> next(it), next(it)
('0', '1')
>>> list(it)
['2', '3', '4', '5', '6', '7', '8', '9']
>>> it.items_seen
10

more_itertools.consumer(func)

Decorator that automatically advances a PEP-342-style “reverse iterator” to its first yield point so you don’t have to call next() on it manually.

>>> @consumer
... def tally():
...     i = 0
...     while True:
...         print('Thing number %s is %s.' % (i, (yield)))
...         i += 1
...
>>> t = tally()
>>> t.send('red')
Thing number 0 is red.
>>> t.send('fish')
Thing number 1 is fish.

Without the decorator, you would have to call next(t) before t.send() could be used.

more_itertools.with_iter(context_manager)

Wrap an iterable in a with statement, so it closes once exhausted.

For example, this will close the file when the iterator is exhausted:

upper_lines = (line.upper() for line in with_iter(open('foo')))

Any context manager which returns an iterable is a candidate for with_iter.

class more_itertools.callback_iter(func, callback_kwd='callback', wait_seconds=0.1)

Convert a function that uses callbacks to an iterator.

Let func be a function that takes a callback keyword argument. For example:

>>> def func(callback=None):
...     for i, c in [(1, 'a'), (2, 'b'), (3, 'c')]:
...         if callback:
...             callback(i, c)
...     return 4

Use with callback_iter(func) to get an iterator over the parameters that are delivered to the callback.

>>> with callback_iter(func) as it:
...     for args, kwargs in it:
...         print(args)
(1, 'a')
(2, 'b')
(3, 'c')

The function will be called in a background thread. The done property indicates whether it has completed execution.

>>> it.done
True

If it completes successfully, its return value will be available in the result property.

>>> it.result
4

Notes:

• If the function uses some keyword argument besides callback, supply callback_kwd.
• If it finished executing, but raised an exception, accessing the result property will raise the same exception.
• If it hasn’t finished executing, accessing the result property from within the with block will raise RuntimeError.
• If it hasn’t finished executing, accessing the result property from outside the with block will raise a more_itertools.AbortThread exception.
• Provide wait_seconds to adjust how frequently the it is polled for output.

---

Itertools recipes

more_itertools.iter_except(func, exception, first=None)

Yields results from a function repeatedly until an exception is raised.

Converts a call-until-exception interface to an iterator interface. Like iter(func, sentinel), but uses an exception instead of a sentinel to end the loop.

>>> l = [0, 1, 2]
>>> list(iter_except(l.pop, IndexError))
[2, 1, 0]

Multiple exceptions can be specified as a stopping condition:

>>> l = [1, 2, 3, '...', 4, 5, 6]
>>> list(iter_except(lambda: 1 + l.pop(), (IndexError, TypeError)))
[7, 6, 5]
>>> list(iter_except(lambda: 1 + l.pop(), (IndexError, TypeError)))
[4, 3, 2]
>>> list(iter_except(lambda: 1 + l.pop(), (IndexError, TypeError)))
[]

Others

New itertools

more_itertools.locate(iterable, pred=bool, window_size=None)

Yield the index of each item in iterable for which pred returns True.

pred defaults to bool(), which will select truthy items:

>>> list(locate([0, 1, 1, 0, 1, 0, 0]))
[1, 2, 4]

Set pred to a custom function to, e.g., find the indexes for a particular item.

>>> list(locate(['a', 'b', 'c', 'b'], lambda x: x == 'b'))
[1, 3]

If window_size is given, then the pred function will be called with that many items. This enables searching for sub-sequences:

>>> iterable = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]
>>> pred = lambda *args: args == (1, 2, 3)
>>> list(locate(iterable, pred=pred, window_size=3))
[1, 5, 9]

Use with seekable() to find indexes and then retrieve the associated items:

>>> from itertools import count
>>> from more_itertools import seekable
>>> source = (3 * n + 1 if (n % 2) else n // 2 for n in count())
>>> it = seekable(source)
>>> pred = lambda x: x > 100
>>> indexes = locate(it, pred=pred)
>>> i = next(indexes)
>>> it.seek(i)
>>> next(it)
106

more_itertools.rlocate(iterable, pred=bool, window_size=None)

Yield the index of each item in iterable for which pred returns True, starting from the right and moving left.

pred defaults to bool(), which will select truthy items:

>>> list(rlocate([0, 1, 1, 0, 1, 0, 0]))  # Truthy at 1, 2, and 4
[4, 2, 1]

Set pred to a custom function to, e.g., find the indexes for a particular item:

>>> iterable = iter('abcb')
>>> pred = lambda x: x == 'b'
>>> list(rlocate(iterable, pred))
[3, 1]

If window_size is given, then the pred function will be called with that many items. This enables searching for sub-sequences:

>>> iterable = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]
>>> pred = lambda *args: args == (1, 2, 3)
>>> list(rlocate(iterable, pred=pred, window_size=3))
[9, 5, 1]

Beware, this function won’t return anything for infinite iterables. If iterable is reversible, rlocate will reverse it and search from the right. Otherwise, it will search from the left and return the results in reverse order.

See locate() to for other example applications.

more_itertools.replace(iterable, pred, substitutes, count=None, window_size=1)

Yield the items from iterable, replacing the items for which pred returns True with the items from the iterable substitutes.

>>> iterable = [1, 1, 0, 1, 1, 0, 1, 1]
>>> pred = lambda x: x == 0
>>> substitutes = (2, 3)
>>> list(replace(iterable, pred, substitutes))
[1, 1, 2, 3, 1, 1, 2, 3, 1, 1]

If count is given, the number of replacements will be limited:

>>> iterable = [1, 1, 0, 1, 1, 0, 1, 1, 0]
>>> pred = lambda x: x == 0
>>> substitutes = [None]
>>> list(replace(iterable, pred, substitutes, count=2))
[1, 1, None, 1, 1, None, 1, 1, 0]

Use window_size to control the number of items passed as arguments to pred. This allows for locating and replacing subsequences.

>>> iterable = [0, 1, 2, 5, 0, 1, 2, 5]
>>> window_size = 3
>>> pred = lambda *args: args == (0, 1, 2)  # 3 items passed to pred
>>> substitutes = [3, 4] # Splice in these items
>>> list(replace(iterable, pred, substitutes, window_size=window_size))
[3, 4, 5, 3, 4, 5]

more_itertools.numeric_range(stop)

more_itertools.numeric_range(start, stop[, step])

An extension of the built-in range() function whose arguments can be any orderable numeric type.

With only stop specified, start defaults to 0 and step defaults to 1. The output items will match the type of stop:

>>> list(numeric_range(3.5))
[0.0, 1.0, 2.0, 3.0]

With only start and stop specified, step defaults to 1. The output items will match the type of start:

>>> from decimal import Decimal
>>> start = Decimal('2.1')
>>> stop = Decimal('5.1')
>>> list(numeric_range(start, stop))
[Decimal('2.1'), Decimal('3.1'), Decimal('4.1')]

With start, stop, and step specified the output items will match the type of start + step:

>>> from fractions import Fraction
>>> start = Fraction(1, 2)  # Start at 1/2
>>> stop = Fraction(5, 2)  # End at 5/2
>>> step = Fraction(1, 2)  # Count by 1/2
>>> list(numeric_range(start, stop, step))
[Fraction(1, 2), Fraction(1, 1), Fraction(3, 2), Fraction(2, 1)]

If step is zero, ValueError is raised. Negative steps are supported:

>>> list(numeric_range(3, -1, -1.0))
[3.0, 2.0, 1.0, 0.0]

Be aware of the limitations of floating point numbers; the representation of the yielded numbers may be surprising.

datetime.datetime objects can be used for start and stop, if step is a datetime.timedelta object:

>>> import datetime
>>> start = datetime.datetime(2019, 1, 1)
>>> stop = datetime.datetime(2019, 1, 3)
>>> step = datetime.timedelta(days=1)
>>> items = iter(numeric_range(start, stop, step))
>>> next(items)
datetime.datetime(2019, 1, 1, 0, 0)
>>> next(items)
datetime.datetime(2019, 1, 2, 0, 0)

more_itertools.side_effect(func, iterable, chunk_size=None, before=None, after=None)

Invoke func on each item in iterable (or on each chunk_size group of items) before yielding the item.

func must be a function that takes a single argument. Its return value will be discarded.

before and after are optional functions that take no arguments. They will be executed before iteration starts and after it ends, respectively.

side_effect can be used for logging, updating progress bars, or anything that is not functionally “pure.”

Emitting a status message:

>>> from more_itertools import consume
>>> func = lambda item: print('Received {}'.format(item))
>>> consume(side_effect(func, range(2)))
Received 0
Received 1

Operating on chunks of items:

>>> pair_sums = []
>>> func = lambda chunk: pair_sums.append(sum(chunk))
>>> list(side_effect(func, [0, 1, 2, 3, 4, 5], 2))
[0, 1, 2, 3, 4, 5]
>>> list(pair_sums)
[1, 5, 9]

Writing to a file-like object:

>>> from io import StringIO
>>> from more_itertools import consume
>>> f = StringIO()
>>> func = lambda x: print(x, file=f)
>>> before = lambda: print(u'HEADER', file=f)
>>> after = f.close
>>> it = [u'a', u'b', u'c']
>>> consume(side_effect(func, it, before=before, after=after))
>>> f.closed
True

more_itertools.iterate(func, start)

Return start, func(start), func(func(start)), …

>>> from itertools import islice
>>> list(islice(iterate(lambda x: 2*x, 1), 10))
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512]

more_itertools.difference(iterable, func=operator.sub, *, initial=None)

This function is the inverse of itertools.accumulate(). By default it will compute the first difference of iterable using operator.sub():

>>> from itertools import accumulate
>>> iterable = accumulate([0, 1, 2, 3, 4])  # produces 0, 1, 3, 6, 10
>>> list(difference(iterable))
[0, 1, 2, 3, 4]

func defaults to operator.sub(), but other functions can be specified. They will be applied as follows:

A, B, C, D, ... --> A, func(B, A), func(C, B), func(D, C), ...

For example, to do progressive division:

>>> iterable = [1, 2, 6, 24, 120]
>>> func = lambda x, y: x // y
>>> list(difference(iterable, func))
[1, 2, 3, 4, 5]

If the initial keyword is set, the first element will be skipped when computing successive differences.

>>> it = [10, 11, 13, 16]  # from accumulate([1, 2, 3], initial=10)
>>> list(difference(it, initial=10))
[1, 2, 3]

more_itertools.make_decorator(wrapping_func, result_index=0)

Return a decorator version of wrapping_func, which is a function that modifies an iterable. result_index is the position in that function’s signature where the iterable goes.

This lets you use itertools on the “production end,” i.e. at function definition. This can augment what the function returns without changing the function’s code.

For example, to produce a decorator version of chunked():

>>> from more_itertools import chunked
>>> chunker = make_decorator(chunked, result_index=0)
>>> @chunker(3)
... def iter_range(n):
...     return iter(range(n))
...
>>> list(iter_range(9))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

To only allow truthy items to be returned:

>>> truth_serum = make_decorator(filter, result_index=1)
>>> @truth_serum(bool)
... def boolean_test():
...     return [0, 1, '', ' ', False, True]
...
>>> list(boolean_test())
[1, ' ', True]

The peekable() and seekable() wrappers make for practical decorators:

>>> from more_itertools import peekable
>>> peekable_function = make_decorator(peekable)
>>> @peekable_function()
... def str_range(*args):
...     return (str(x) for x in range(*args))
...
>>> it = str_range(1, 20, 2)
>>> next(it), next(it), next(it)
('1', '3', '5')
>>> it.peek()
'7'
>>> next(it)
'7'

class more_itertools.SequenceView(target)

Return a read-only view of the sequence object target.

SequenceView objects are analogous to Python’s built-in “dictionary view” types. They provide a dynamic view of a sequence’s items, meaning that when the sequence updates, so does the view.

>>> seq = ['0', '1', '2']
>>> view = SequenceView(seq)
>>> view
SequenceView(['0', '1', '2'])
>>> seq.append('3')
>>> view
SequenceView(['0', '1', '2', '3'])

Sequence views support indexing, slicing, and length queries. They act like the underlying sequence, except they don’t allow assignment:

>>> view[1]
'1'
>>> view[1:-1]
['1', '2']
>>> len(view)
4

Sequence views are useful as an alternative to copying, as they don’t require (much) extra storage.

more_itertools.time_limited(limit_seconds, iterable)

Yield items from iterable until limit_seconds have passed. If the time limit expires before all items have been yielded, the timed_out parameter will be set to True.

>>> from time import sleep
>>> def generator():
...     yield 1
...     yield 2
...     sleep(0.2)
...     yield 3
>>> iterable = time_limited(0.1, generator())
>>> list(iterable)
[1, 2]
>>> iterable.timed_out
True

Note that the time is checked before each item is yielded, and iteration stops if the time elapsed is greater than limit_seconds. If your time limit is 1 second, but it takes 2 seconds to generate the first item from the iterable, the function will run for 2 seconds and not yield anything.

more_itertools.map_if(iterable, pred, func, func_else=lambda x: x)

Evaluate each item from iterable using pred. If the result is equivalent to True, transform the item with func and yield it. Otherwise, transform the item with func_else and yield it.

pred, func, and func_else should each be functions that accept one argument. By default, func_else is the identity function.

>>> from math import sqrt
>>> iterable = list(range(-5, 5))
>>> iterable
[-5, -4, -3, -2, -1, 0, 1, 2, 3, 4]
>>> list(map_if(iterable, lambda x: x > 3, lambda x: 'toobig'))
[-5, -4, -3, -2, -1, 0, 1, 2, 3, 'toobig']
>>> list(map_if(iterable, lambda x: x >= 0,
... lambda x: f'{sqrt(x):.2f}', lambda x: None))
[None, None, None, None, None, '0.00', '1.00', '1.41', '1.73', '2.00']

---

Itertools recipes

more_itertools.iter_index(iterable, value, start=0)

Yield the index of each place in iterable that value occurs, beginning with index start.

See locate() for a more general means of finding the indexes associated with particular values.

>>> list(iter_index('AABCADEAF', 'A'))
[0, 1, 4, 7]

more_itertools.consume(iterator, n=None)

Advance iterable by n steps. If n is None, consume it entirely.

Efficiently exhausts an iterator without returning values. Defaults to consuming the whole iterator, but an optional second argument may be provided to limit consumption.

>>> i = (x for x in range(10))
>>> next(i)
0
>>> consume(i, 3)
>>> next(i)
4
>>> consume(i)
>>> next(i)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

If the iterator has fewer items remaining than the provided limit, the whole iterator will be consumed.

>>> i = (x for x in range(3))
>>> consume(i, 5)
>>> next(i)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

more_itertools.tabulate(function, start=0)

Return an iterator over the results of func(start), func(start + 1), func(start + 2)…

func should be a function that accepts one integer argument.

If start is not specified it defaults to 0. It will be incremented each time the iterator is advanced.

>>> square = lambda x: x ** 2
>>> iterator = tabulate(square, -3)
>>> take(4, iterator)
[9, 4, 1, 0]

more_itertools.repeatfunc(func, times=None, *args)

Call func with args repeatedly, returning an iterable over the results.

If times is specified, the iterable will terminate after that many repetitions:

>>> from operator import add
>>> times = 4
>>> args = 3, 5
>>> list(repeatfunc(add, times, *args))
[8, 8, 8, 8]

If times is None the iterable will not terminate:

>>> from random import randrange
>>> times = None
>>> args = 1, 11
>>> take(6, repeatfunc(randrange, times, *args))  # doctest:+SKIP
[2, 4, 8, 1, 8, 4]

more_itertools.polynomial_from_roots(roots)

Compute a polynomial’s coefficients from its roots.

>>> roots = [5, -4, 3]  # (x - 5) * (x + 4) * (x - 3)
>>> polynomial_from_roots(roots)  # x^3 - 4 * x^2 - 17 * x + 60
[1, -4, -17, 60]

more_itertools.sieve(n)

Yield the primes less than n.

>>> list(sieve(30))
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]

more_itertools.factor(n)

Yield the prime factors of n. >>> list(factor(360)) [2, 2, 2, 3, 3, 5]

more_itertools.matmul(m1, m2)

Multiply two matrices. >>> list(matmul([(7, 5), (3, 5)], [(2, 5), (7, 9)])) [[49, 80], [41, 60]]

The caller should ensure that the dimensions of the input matrices are compatible with each other.

License

more-itertools is under the MIT License. See the LICENSE file.

Conditions for Contributors

By contributing to this software project, you are agreeing to the following terms and conditions for your contributions: First, you agree your contributions are submitted under the MIT license. Second, you represent you are authorized to make the contributions and grant the license. If your employer has rights to intellectual property that includes your contributions, you represent that you have received permission to make contributions and grant the required license on behalf of that employer.

Testing

To run install dependencies and run tests, use this command:

python -m unittest

Multiple Python Versions

To run the tests on all the versions of Python more-itertools supports, install tox:

pip install tox

Then, run the tests:

tox

Version History

More routines for operating on iterables, beyond itertools

8.14.0

• New functions

  • iter_index() (from the Python itertools docs)
  • transpose() (from the Python itertools docs)
  • matmul() (from the Python itertools docs)
  • factor() (from the Python itertools docs)
  • gray_product() (thanks to haukex)

• Changes to existing functions

  • sieve() was updated to match the Python itertools docs
  • maxsplit() was updated to fix a bug (thanks to abingham)
  • sliced() had its type hint updated (thanks to ad-chaos)

• Other changes

  • The batched function is marked as deprecated and will be removed in a future major release. For Python 3.12 and above, use itertools.batched instead. (thanks to neutrinoceros)
  • The type hints now used postponed evaluation of annotations from PEP 563 (thanks to Isira-Seneviratne)
  • Some documentation issues were fixed (thanks to Voskov and jdkandersson)

9.0.0

• Potentially breaking changes

  • grouper() no longer accepts an integer as its first argument. Previously this raised a DeprecationWarning.
  • collate() has been removed. Use the built-in heapq.merge() instead.
  • windowed() now yields nothing when its iterable is empty.
  • This library now advertises support for Python 3.7+.

• New functions

  • constrained_batches()
  • batched() (from the Python itertools docs)
  • polynomial_from_roots() (from the Python itertools docs)
  • sieve() (from the Python itertools docs)

• Other changes

  • Some documentation issues were fixed (thanks to nanouasyn)

8.14.0

• New functions

  • longest_common_prefix() (thanks to nanouasyn)
  • iequals() (thanks to nanouasyn)

• Changes to existing functions

  • concurrent.futures.ThreadPoolExecutor is now imported lazily in callback_iter().
  • tail() is now optimized for iterables with a fixed length.

• Other changes

  • Some documentation issues were fixed (thanks to pochmann and timgates42)
  • This library is now marked for Python 3.10 compatibility in PyPI (thanks to chayim)

8.13.0

• New functions

  • The subslices() recipe from the itertools docs was added (thanks to rhettinger)

• Changes to existing functions

  • The ichunked() function is now more efficient (thanks to hjtran0 and seanmacavaney)
  • The difference() function is now more efficient (thanks to Masynchin)
  • The grouper() recipe now has more features, mirroring the one in the itertools docs (thanks to rhettinger)

• Other changes

  • Some documentation issues were fixed (thanks to medvied and Freed-Wu)
  • The more_itertools package is now built with flit (thanks to mgorny)

8.12.0

• Bug fixes

  • Some documentation issues were fixed (thanks to Masynchin, spookylukey, astrojuanlu, and stephengmatthews)
  • Python 3.5 support was temporarily restored (thanks to mattbonnell)

8.11.0

• New functions

  • The before_and_after(), sliding_window(), and triplewise() recipes from the Python 3.10 docs were added
  • duplicates_everseen() and duplicates_justseen() (thanks to OrBin and DavidPratt512)
  • minmax() (thanks to Ricocotam, MSeifert04, and ruancomelli)
  • strictly_n() (thanks to hwalinga and NotWearingPants)
  • unique_in_window()

• Changes to existing functions

  • groupby_transform() had its type stub improved (thanks to mjk4 and ruancomelli)
  • is_sorted() now accepts a strict parameter (thanks to Dutcho and ruancomelli)
  • zip_broadcast() was updated to fix a bug (thanks to kalekundert)

8.10.0

• Changes to existing functions

  • The type stub for iter_except() was improved (thanks to MarcinKonowalczyk)

• Other changes:

  • Type stubs now ship with the source release (thanks to saaketp)
  • The Sphinx docs were improved (thanks to MarcinKonowalczyk)

8.9.0

• New functions

  • interleave_evenly() (thanks to mbugert)
  • repeat_each() (thanks to FinalSh4re)
  • chunked_even() (thanks to valtron)
  • map_if() (thanks to sassbalint)
  • zip_broadcast() (thanks to kalekundert)

• Changes to existing functions

  • The type stub for chunked() was improved (thanks to PhilMacKay)
  • The type stubs for zip_equal() and zip_offset were improved (thanks to maffoo)
  • Building Sphinx docs locally was improved (thanks to MarcinKonowalczyk)

8.8.0

• New functions

  • countable() (thanks to krzysieq)

• Changes to existing functions

  • split_before() was updated to handle empy collections (thanks to TiunovNN)
  • unique_everseen() got a performance boost (thanks to Numerlor)
  • The type hint for value_chain() was corrected (thanks to vr2262)

8.7.0

• New functions

  • convolve() (from the Python itertools docs)
  • product_index(), combination_index(), and permutation_index() (thanks to N8Brooks)
  • value_chain() (thanks to jenstroeger)

• Changes to existing functions

  • distinct_combinations() now uses a non-recursive algorithm (thanks to knutdrand)
  • pad_none() is now the preferred name for padnone(), though the latter remains available.
  • pairwise() will now use the Python standard library implementation on Python 3.10+
  • sort_together() now accepts a key argument (thanks to brianmaissy)
  • seekable() now has a peek method, and can indicate whether the iterator it’s wrapping is exhausted (thanks to gsakkis)
  • time_limited() can now indicate whether its iterator has expired (thanks to roysmith)
  • The implementation of unique_everseen() was improved (thanks to plammens)

• Other changes:

  • Various documentation updates (thanks to cthoyt, Evantm, and cyphase)

8.6.0

• New itertools

  • all_unique() (thanks to brianmaissy)
  • nth_product() and nth_permutation() (thanks to N8Brooks)

• Changes to existing itertools

  • chunked() and sliced() now accept a strict parameter (thanks to shlomif and jtwool)

• Other changes

  • Python 3.5 has reached its end of life and is no longer supported.
  • Python 3.9 is officially supported.
  • Various documentation fixes (thanks to timgates42)

8.5.0

• New itertools

  • windowed_complete() (thanks to MarcinKonowalczyk)

• Changes to existing itertools:

  • The is_sorted() implementation was improved (thanks to cool-RR)
  • The groupby_transform() now accepts a reducefunc parameter.
  • The last() implementation was improved (thanks to brianmaissy)

• Other changes

  • Various documentation fixes (thanks to craigrosie, samuelstjean, PiCT0)
  • The tests for distinct_combinations() were improved (thanks to Minabsapi)

  • Automated tests now run on GitHub Actions. All commits now check:

    • That unit tests pass
    • That the examples in docstrings work
    • That test coverage remains high (using coverage)
    • For linting errors (using flake8)
    • For consistent style (using black)
    • That the type stubs work (using mypy)
    • That the docs build correctly (using sphinx)
    • That packages build correctly (using twine)

8.4.0

• New itertools

  • mark_ends() (thanks to kalekundert)
  • is_sorted()

• Changes to existing itertools:

  • islice_extended() can now be used with real slices (thanks to cool-RR)
  • The implementations for filter_except() and map_except() were improved (thanks to SergBobrovsky)

• Other changes

  • Automated tests now enforce code style (using black)
  • The various signatures of islice_extended() and numeric_range() now appear in the docs (thanks to dsfulf)
  • The test configuration for mypy was updated (thanks to blueyed)

8.3.0

• New itertools

  • zip_equal() (thanks to frankier and alexmojaki)

• Changes to existing itertools:

  • split_at(), split_before(), split_after(), and split_when() all got a maxsplit paramter (thanks to jferard and ilai-deutel)
  • split_at() now accepts a keep_separator parameter (thanks to jferard)
  • distinct_permutations() can now generate r-length permutations (thanks to SergBobrovsky and ilai-deutel)
  • The windowed() implementation was improved (thanks to SergBobrovsky)
  • The spy() implementation was improved (thanks to has2k1)

• Other changes

  • Type stubs are now tested with stubtest (thanks to ilai-deutel)
  • Tests now run with python -m unittest instead of python setup.py test (thanks to jdufresne)

8.2.0

• Bug fixes

  • The .pyi files for typing were updated. (thanks to blueyed and ilai-deutel)

• Changes to existing itertools:

  • numeric_range() now behaves more like the built-in range(). (thanks to jferard)
  • bucket() now allows for enumerating keys. (thanks to alexchandel)
  • sliced() now should now work for numpy arrays. (thanks to sswingle)
  • seekable() now has a maxlen parameter.

8.1.0

• Bug fixes

  • partition() works with pred=None again. (thanks to MSeifert04)

• New itertools

  • sample() (thanks to tommyod)
  • nth_or_last() (thanks to d-ryzhikov)

• Changes to existing itertools:

  • The implementation for divide() was improved. (thanks to jferard)

8.0.2

• Bug fixes

  • The type stub files are now part of the wheel distribution (thanks to keisheiled)

8.0.1

• Bug fixes

  • The type stub files now work for functions imported from the root package (thanks to keisheiled)

8.0.0

• New itertools and other additions

  • This library now ships type hints for use with mypy. (thanks to ilai-deutel for the implementation, and to gabbard and fmagin for assistance)
  • split_when() (thanks to jferard)
  • repeat_last() (thanks to d-ryzhikov)

• Changes to existing itertools:

  • The implementation for set_partitions() was improved. (thanks to jferard)
  • partition() was optimized for expensive predicates. (thanks to stevecj)
  • unique_everseen() and groupby_transform() were re-factored. (thanks to SergBobrovsky)
  • The implementation for difference() was improved. (thanks to Jabbey92)

• Other changes

  • Python 3.4 has reached its end of life and is no longer supported.
  • Python 3.8 is officially supported. (thanks to jdufresne)
  • The collate function has been deprecated. It raises a DeprecationWarning if used, and will be removed in a future release.
  • one() and only() now provide more informative error messages. (thanks to gabbard)
  • Unit tests were moved outside of the main package (thanks to jdufresne)
  • Various documentation fixes (thanks to kriomant, gabbard, jdufresne)

7.2.0

• New itertools

  • distinct_combinations()
  • set_partitions() (thanks to kbarrett)
  • filter_except()
  • map_except()

7.1.0

• New itertools

  • ichunked() (thanks davebelais and youtux)
  • only() (thanks jaraco)

• Changes to existing itertools:

  • numeric_range() now supports ranges specified by datetime.datetime and datetime.timedelta objects (thanks to MSeifert04 for tests).
  • difference() now supports an initial keyword argument.

• Other changes

  • Various documentation fixes (thanks raimon49, pylang)

7.0.0

• New itertools:

  • time_limited()
  • partitions() (thanks to rominf and Saluev)
  • substrings_indexes() (thanks to rominf)

• Changes to existing itertools:

  • collapse() now treats bytes objects the same as str objects. (thanks to Sweenpet)

The major version update is due to the change in the default behavior of collapse(). It now treats bytes objects the same as str objects. This aligns its behavior with always_iterable().

>>> from more_itertools import collapse
>>> iterable = [[1, 2], b'345', [6]]
>>> print(list(collapse(iterable)))
[1, 2, b'345', 6]

6.0.0

• Major changes:

  • Python 2.7 is no longer supported. The 5.0.0 release will be the last version targeting Python 2.7.
  • All future releases will target the active versions of Python 3. As of 2019, those are Python 3.4 and above.
  • The six library is no longer a dependency.
  • The accumulate() function is no longer part of this library. You may import a better version from the standard itertools module.

• Changes to existing itertools:

  • The order of the parameters in grouper() have changed to match the latest recipe in the itertools documentation. Use of the old order will be supported in this release, but emit a DeprecationWarning. The legacy behavior will be dropped in a future release. (thanks to jaraco)
  • distinct_permutations() was improved (thanks to jferard - see also permutations with unique values at StackOverflow.)
  • An unused parameter was removed from substrings(). (thanks to pylang)

• Other changes:

  • The docs for unique_everseen() were improved. (thanks to jferard and MSeifert04)
  • Several Python 2-isms were removed. (thanks to jaraco, MSeifert04, and hugovk)

5.0.0

• New itertools:

  • split_into() (thanks to rovyko)
  • unzip() (thanks to bmintz)
  • substrings() (thanks to pylang)

• Changes to existing itertools:

  • ilen() was optimized a bit (thanks to MSeifert04, achampion, and bmintz)
  • first_true() now returns None by default. This is the reason for the major version bump - see below. (thanks to sk and OJFord)

• Other changes:

  • Some code for old Python versions was removed (thanks to hugovk)
  • Some documentation mistakes were corrected (thanks to belm0 and hugovk)
  • Tests now run properly on 32-bit versions of Python (thanks to Millak)
  • Newer versions of CPython and PyPy are now tested against

The major version update is due to the change in the default return value of first_true(). It’s now None.

>>> from more_itertools import first_true
>>> iterable = [0, '', False, [], ()]  # All these are False
>>> answer = first_true(iterable)
>>> print(answer)
None

4.3.0

• New itertools:

  • last() (thanks to tmshn)
  • replace() (thanks to pylang)
  • rlocate() (thanks to jferard and pylang)

• Improvements to existing itertools:

  • locate() can now search for multiple items

• Other changes:

  • The docs now include a nice table of tools (thanks MSeifert04)

4.2.0

• New itertools:

  • map_reduce() (thanks to pylang)
  • prepend() (from the Python 3.7 docs)

• Improvements to existing itertools:

  • bucket() now complies with PEP 479 (thanks to irmen)

• Other changes:

  • Python 3.7 is now supported (thanks to irmen)
  • Python 3.3 is no longer supported
  • The test suite no longer requires third-party modules to run
  • The API docs now include links to source code

4.1.0

• New itertools:

  • split_at() (thanks to michael-celani)
  • circular_shifts() (thanks to hiqua)
  • make_decorator() - see the blog post Yo, I heard you like decorators for a tour (thanks to pylang)
  • always_reversible() (thanks to michael-celani)
  • nth_combination() (from the Python 3.7 docs)

• Improvements to existing itertools:

  • seekable() now has an elements method to return cached items.
  • The performance tradeoffs between roundrobin() and interleave_longest() are now documented (thanks michael-celani, pylang, and MSeifert04)

4.0.1

• No code changes - this release fixes how the docs display on PyPI.

4.0.0

• New itertools:

  • consecutive_groups() (Based on the example in the Python 2.4 docs)
  • seekable() (If you’re looking for how to “reset” an iterator, you’re in luck!)
  • exactly_n() (thanks to michael-celani)
  • run_length.encode() and run_length.decode()
  • difference()

• Improvements to existing itertools:

  • The number of items between filler elements in intersperse() can now be specified (thanks to pylang)
  • distinct_permutations() and peekable() got some minor adjustments (thanks to MSeifert04)
  • always_iterable() now returns an iterator object. It also now allows different types to be considered iterable (thanks to jaraco)
  • bucket() can now limit the keys it stores in memory
  • one() now allows for custom exceptions (thanks to kalekundert)

• Other changes:

  • A few typos were fixed (thanks to EdwardBetts)
  • All tests can now be run with python setup.py test

The major version update is due to the change in the return value of always_iterable(). It now always returns iterator objects:

>>> from more_itertools import always_iterable
# Non-iterable objects are wrapped with iter(tuple(obj))
>>> always_iterable(12345)
<tuple_iterator object at 0x7fb24c9488d0>
>>> list(always_iterable(12345))
[12345]
# Iterable objects are wrapped with iter()
>>> always_iterable([1, 2, 3, 4, 5])
<list_iterator object at 0x7fb24c948c50>

3.2.0

• New itertools:

  • lstrip(), rstrip(), and strip() (thanks to MSeifert04 and pylang)
  • islice_extended()

• Improvements to existing itertools:

  • Some bugs with slicing peekable()-wrapped iterables were fixed

3.1.0

• New itertools:

  • numeric_range() (Thanks to BebeSparkelSparkel and MSeifert04)
  • count_cycle() (Thanks to BebeSparkelSparkel)
  • locate() (Thanks to pylang and MSeifert04)

• Improvements to existing itertools:

  • A few itertools are now slightly faster due to some function optimizations. (Thanks to MSeifert04)

• The docs have been substantially revised with installation notes, categories for library functions, links, and more. (Thanks to pylang)

3.0.0

• Removed itertools:

  • context has been removed due to a design flaw - see below for replacement options. (thanks to NeilGirdhar)

• Improvements to existing itertools:

  • side_effect now supports before and after keyword arguments. (Thanks to yardsale8)

• PyPy and PyPy3 are now supported.

The major version change is due to the removal of the context function. Replace it with standard with statement context management:

# Don't use context() anymore
file_obj = StringIO()
consume(print(x, file=f) for f in context(file_obj) for x in u'123')

# Use a with statement instead
file_obj = StringIO()
with file_obj as f:
    consume(print(x, file=f) for x in u'123')

2.6.0

• New itertools:

  • adjacent and groupby_transform (Thanks to diazona)
  • always_iterable (Thanks to jaraco)
  • (Removed in 3.0.0) context (Thanks to yardsale8)
  • divide (Thanks to mozbhearsum)

• Improvements to existing itertools:

  • ilen is now slightly faster. (Thanks to wbolster)
  • peekable can now prepend items to an iterable. (Thanks to diazona)

2.5.0

• New itertools:

  • distribute (Thanks to mozbhearsum and coady)
  • sort_together (Thanks to clintval)
  • stagger and zip_offset (Thanks to joshbode)
  • padded

• Improvements to existing itertools:

  • peekable now handles negative indexes and slices with negative components properly.
  • intersperse is now slightly faster. (Thanks to pylang)
  • windowed now accepts a step keyword argument. (Thanks to pylang)

• Python 3.6 is now supported.

2.4.1

• Move docs 100% to readthedocs.io.

2.4

• New itertools:

  • accumulate, all_equal, first_true, partition, and tail from the itertools documentation.
  • bucket (Thanks to Rosuav and cvrebert)
  • collapse (Thanks to abarnet)
  • interleave and interleave_longest (Thanks to abarnet)
  • side_effect (Thanks to nvie)
  • sliced (Thanks to j4mie and coady)
  • split_before and split_after (Thanks to astronouth7303)
  • spy (Thanks to themiurgo and mathieulongtin)

• Improvements to existing itertools:

  • chunked is now simpler and more friendly to garbage collection. (Contributed by coady, with thanks to piskvorky)
  • collate now delegates to heapq.merge when possible. (Thanks to kmike and julianpistorius)
  • peekable-wrapped iterables are now indexable and sliceable. Iterating through peekable-wrapped iterables is also faster.
  • one and unique_to_each have been simplified. (Thanks to coady)

2.3

• Added one from jaraco.util.itertools. (Thanks, jaraco!)
• Added distinct_permutations and unique_to_each. (Contributed by bbayles)
• Added windowed. (Contributed by bbayles, with thanks to buchanae, jaraco, and abarnert)
• Simplified the implementation of chunked. (Thanks, nvie!)
• Python 3.5 is now supported. Python 2.6 is no longer supported.
• Python 3 is now supported directly; there is no 2to3 step.

2.2

• Added iterate and with_iter. (Thanks, abarnert!)

2.1

• Added (tested!) implementations of the recipes from the itertools documentation. (Thanks, Chris Lonnen!)
• Added ilen. (Thanks for the inspiration, Matt Basta!)

2.0

• chunked now returns lists rather than tuples. After all, they’re homogeneous. This slightly backward-incompatible change is the reason for the major version bump.
• Added @consumer.
• Improved test machinery.

1.1

• Added first function.
• Added Python 3 support.
• Added a default arg to peekable.peek().
• Noted how to easily test whether a peekable iterator is exhausted.
• Rewrote documentation.

1.0

• Initial release, with collate, peekable, and chunked. Could really use better docs.