Locked learning resources

Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Locked learning resources

This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Functions: Iterables and Iterators

00:01 Iterables and Iterators. This section is going to look at functions you can use with iterables and iterators. The first one is going to be len(), which tells you the length of the iterable that you pass it.

00:18 Here’s a list filled with some random-ish numbers. Looking at len(a) shows us there are 9 elements within that list. We can see that that list does indeed have nine elements. Next up, any() and all().

00:36 any() and all() work on the Boolean values of the iterables that you pass to them. As we’ve seen earlier, any entry can be considered to be True or False, but to simplify matters, the three lists are going to explicitly use True or False. So in the case of a, all three entries are True.

00:58 In the case of b, some of the entries are True but it has a False present as well.

01:08 And the third list will have all False values. So these are the three possibilities which are present.

01:19 If we look at any(), we can see that any(a) is True because some of them are True. In fact, all of them are True. If we look at any(b), that’s also True because it has some True values even though they’re not all True.

01:36 Looking at any(c) gives us the result False because none of them are True.

01:45 The partner of this is all(). If we look at all(a), because they’re all True we get True, as you can see here.

01:55 all(b) gives us False, because not all of them are because we have one False value there. And as you can imagine, for all(c) we also get False because there are no True values, let alone all of them being True.

02:16 For the next section, we’re going to define a variable d as a list of numbers. As you can see, it has nine elements in it.

02:30 Let’s look at the reversed() function. If we define e as reversed(d), we’d hoped to get that list out backwards, and if we look at it, we don’t get exactly what we’re expecting.

02:43 What you may have been expecting is a reverse version of that list, and instead we get a reverse iterator object. Now, most of the time in your code, that will be okay, but some of the time you may want to see that list—for instance, when you’re debugging or printing it out. So if we do list(e), you can see that list is d in reverse.

03:04 But that works how you probably expect, and sometimes you will need to do this, but often within your code, you won’t need to do this because Python will deal with the iterator.

03:15 Instead of casting all of this to a list in one go and outputting it, Python will do it in a more efficient manner and only produce the value which is needed when they’re needed.

03:26 The analogy I can think of is imagine having a machine which could produce cars for you. You’d have two modes it could operate in—one where it produces all the cars you need in one go, and one where it does them on demand.

03:40 That would be fine if you only needed one, but if you needed 3 million of them, you’d need to find somewhere to store them all. In this case, it’s storing all those numbers in memory, and it takes time to create them.

03:52 Generally, Python’s create-on-demand system is a more efficient way for it to work and has much lower memory demands, but sometimes dealing with this when you’re starting out means you need to cast things into a list to understand what’s happening behind the scenes. With that out of the way, let’s look at sorted(). Here we’re defining f as the sorted version of d.

04:13 If we look at f, it works the way we’d expect—it produces a list and it’s the elements of d here which have been sorted. And see, we’ve got two 7s, which is why we’ve got two 7s there.

04:27 This list is exactly as we’d expect. And it’s also possible to reverse sorted() if you need it to be that way around.

04:37 Next up, let’s look at range(). Now, range() is really useful. You may have already encountered it. If we enter range() here in the REPL, we get this range from 0 to 10, which isn’t that helpful because it doesn’t really tell us what’s going on, but if we cast that into a list, we can see we get the values 0 to 9.

05:00 Remember, it’s not including this final number, so it’s 0 to 9we get 10 in total. It’s possible to pass it explicitly the start and end values, and also a third value, stride.

05:14 Here we can see a range(0, 10) with a stride of 2. That gives just the even numbers, 0 to 8, and again, it doesn’t include the last number. range() is extremely useful and you’ve probably used it already, but you may not have heard of these last two, zip() and enumerate(), which are easier to understand in the context of a program, so we’re going to switch over to code now.

05:40 Iterables and Iterators: enumerate(). As a demonstration of this, a simple program is going to be created which is going to print out the list of players of an imaginary team.

05:55 A typical thing, we have a list here and I’m going to create a for loop. for player in players: print(player). Fairly piecemeal stuff, and you’ve probably seen something like this before.

06:08 If we see that in action, it works exactly as we’d expect, printing each name on its own line. But quite often it’s important to know which number on the list is which index in the list. The way you would probably do this if you weren’t aware of enumerate() is to create a count variable which is initialized before the loop starts.

06:31 We’ll print the count as well as the player out on each line, and then after that, the count gets incremented with count += 1, and that’ll work just fine as we can see here.

06:43 But Python provides us with enumerate(), which makes a much cleaner way to do this same functionality. We’re going to delete the count variable, we’re going to delete the increment of count, and then we’re going to wrap players in the enumerate() function, which outputs two things. It outputs the normal player, but it will also output the count—a zero-indexed counter of the number of times a loop has run. So we have much neater code, you don’t have to worry about initializing or incrementing the count variable—which everyone has forgotten at some point—and running the code gives us exactly the same output as before, but it’s a much neater piece of code to read.

07:24 It’s easy to see exactly what’s happening, and because it’s standard code, you’ll get used to seeing this and understand it intuitively every time.

07:38 Iterables and Iterators: zip().

07:42 In this example, you have two lists which need to be paired together—so in this case, a list of three countries,

07:57 and one which is continents which go with them.

08:07 You’ll want to create tuples with these paired together. The way you would initially do this would probably involve a for loop, so in this case, using an index variable combining range() and the len() of the countries list.

08:23 That index will now allow us to access the countries[i] and continents[i].

08:36 You can check if that works by running the program,

08:41 and as you can see, they’ve been paired correctly: 'France' with 'Europe', 'Tanzania' with 'Africa', 'Canada' with 'North America'.

08:49 Now that we know we’re accessing those correctly, we can put these together. So first things first, let’s create a list for them to go in. Here’s our empty merged list,

09:04 and then we can do merged.append() and put that in brackets inside there so we create a tuple with those two together.

09:15 Finally, we can print out the merged list, and now we can see that running and we have our list with the appropriately paired countries and continents.

09:29 If you’re not aware of the zip() function, this is probably the kind of thing you would do many times. For there’s a much simpler way to do this, and as we can see, we can use merged_2, it uses the zip() function with those two lists countries and continents, and that’s that entire for loop replaced.

09:51 When we run this, they both produce identical output, but this is a much more complicated equivalent to this. The zip() part is much simpler than these three lines where you create an empty list and a for loop. It could have been done with a list comprehension but still would have been complicated. But zip() works faster, is much easier, and is much less error-prone than the manual approach.

10:17 It’s important to remember that this is a zip in terms of a zipper on your clothes, not in terms of a compressed file creator. zip() is incredibly useful whenever you have lists you need to merge together.

10:30 Iterables and Iterators: iter() and next(). The last thing we’re going to look at in this section is the creation of an iterator such as we saw earlier on. It’s possible to create an iterator using any iterable. We’re going to use a list in this example.

10:47 It has a method next(), which pops the next value off of that iterator until it’s exhausted, when you’ll get a StopIteration exception. Let’s make an example now, so here a = iter() and we’re just going to pass it a list directly of [1, 2, 3, 4, 5, 6], to keep it nice and simple.

11:12 Now we’ve created our iterator a and now we can just use next(a) and we get the next value from it, and then 2, 3, 4, 5.

11:21 And once it’s exhausted, if we try next() again, we get a StopIteration exception. This is the way that any iterable works behind the scenes with Python, so anything which can take an iterator will work exactly the same way with an iterator you’ve created using the iter() keyword.

Avatar image for alanhrosenthal

alanhrosenthal on March 21, 2020

Very complete. Very easy to understand.

Well Done!

Avatar image for Mahesh

Mahesh on March 25, 2020

getting below error for ZIP.py <zip object at 0x00633F80>

Avatar image for peterabbeel

peterabbeel on March 29, 2020

Related to above comment:

getting below error for ZIP.py <zip object at 0x00633F80>.

Note this is not an error.

print(merged_2) returns the memory address of the iterator merged_2 print(list(merged_2)) returns the content of the iterator merged_2

You can also call the zip functionality as following:

print(list(zip(countries, continents)))
Avatar image for Cory

Cory on April 24, 2020

Exactly what peterabbeel did on <zip object at 0x036B4F08> I also had to switch mine to print(tuple(merged_2) which then gave me the proper output although it makes it immutable. Still learned a ton from this tutorial.

Avatar image for Alain Rouleau

Alain Rouleau on Aug. 7, 2020

I didn’t even know some of these functions existed such as any() and all(). Of course everybody should already know about them since their just built-in functions. But very handy.

Learned quite a lot from this video, thanks.

Avatar image for leroygcooper

leroygcooper on Nov. 26, 2020

Hey Darren. In the zip function example using the “merged” for-loop, were there two sets of brackets around the “merged.append…” line due to the argument being a tuple (countries[i], continents[i])?

Avatar image for Bartosz Zaczyński

Bartosz Zaczyński RP Team on Nov. 26, 2020

@leroygcooper That’s correct. You need to wrap both elements in a tuple because the list’s .append() method expects only a single item, and we want the list to contain pairs.

Avatar image for catchkarthik

catchkarthik on Jan. 30, 2022

Can you give me an actual use case where function reversed() can be used? I understand that python stores it as an object and we have to feed it to a list to retrieve the values. Is there a use case where reversed can be used without a list function? may be in a loop function where i need to print the items in a list, in a reverse order?

Avatar image for Bartosz Zaczyński

Bartosz Zaczyński RP Team on Jan. 31, 2022

@catchkarthik The reversed() function accepts any iterable object, while the list.reverse() method is only defined for lists. Additionally, sequences like lists, tuples, or strings support the slicing syntax, e.g., text[::-1], which lets you achieve a similar effect by making a reversed copy of the original sequence. On the other hand, reversed() is lazily evaluated and creates a lightweight view of the original data without allocating new memory. This makes changes to the original data visible in your reversed view. You’ll find much more details about the differences between those methods for reversing sequences in the written tutorial: realpython.com/python-reverse-list/

You’d use reversed() in a loop or a comprehension expression, like you said. There’s also a protocol you can leverage in your custom classes:

>>> class CustomClass:
...     def __iter__(self):
...         yield from [1, 2, 3]
...     
...     def __reversed__(self):
...         yield from [3, 2, 1]

>>> instance = CustomClass()
>>> for value in instance:
...     print(value)
...
1
2
3

>>> for value in reversed(instance):
...     print(value)
...
3
2
1
Avatar image for aquibmir

aquibmir on March 2, 2022

So what exactly is the difference between an Iterable and an Iterator? How do we define them?

e = reversed(d)

is the Iterator and

list(e)

is the Iterable?

Avatar image for Bartosz Zaczyński

Bartosz Zaczyński RP Team on March 2, 2022

@aquibmir See the Python Glossary for the relevant definitions:

In a nutshell, an iterable is a composite object able to yield its elements in a for-loop. There are all kinds of iterable objects in Python, including strings, tuples, lists, dictionaries, and so on. On the other hand, objects like integers aren’t iterable because you can’t iterate over them in a loop.

An iterator is another type of object whose role is to hide the details of visiting the individual elements of an iterable. It’s straightforward to think of iterating over a sequence like a Python list because each element has an associated index. However, some iterables don’t keep their elements in any particular order. For example, sets, trees, graphs, and other data structures are organized differently in memory, yet they all behave the same when you iterate over them.

You can define your own iterable that hooks into the for-loop by implementing a few special methods in a custom class. If your iterable has a known length and is a sequence indexed by a numeric index or a mapping accessed by a hashable key, then you can implement these two special methods in your custom class:

class Iterable:
    def __getitem__(self, key_or_index):
        ...
    def __len__(self):
        ...

Otherwise, you can use a more generic iterator protocol by implementing the .__iter__() special method instead:

class Iterable:
    def __iter__(self):
        return Iterator(...)

Moreover, if you’d like to support iteration in reversed order, i.e., when someone calls reversed() on your iterable, then you can optionally add yet another special method:

class Iterable:
    def __iter__(self):
        return Iterator(...)
    def __reversed__(self):
        return ReversedIterator(...)

In both cases, you return a new instance of an iterator class, which will handle the iteration. Alternatively, when the iteration has a relatively simple logic, it’s customary to use a generator iterator instead of returning a new instance of a class:

class Iterable1:
    def __iter__(self):
        yield from self.items

class Iterable2:
    def __iter__(self):
        for item in self.items:
            yield item

If you choose to define an iterator class, then it must implement at least the .__next__() special method, which supplies the next element of the iterable. However, to comply with the iterator protocol, it should also implement the .__iter__() method that returns itself:

class Iterator:
    def __next__(self):
        ...
    def __iter__(self):
        return self

This makes the iterator iterable itself, which is mandatory if you want to use it with a for-loop, which always calls iter() first, and then keeps calling next() on the resulting iterator object. To test your iterable and the corresponding iterator, you can simulate a for-loop by calling these functions manually:

>>> class Iterable:
...     def __init__(self, *elements):
...         self.elements = elements
...     def __iter__(self):
...         return Iterator(self.elements)

>>> class Iterator:
...     def __init__(self, elements):
...         self.elements = elements
...         self.index = -1
...     def __next__(self):
...         self.index += 1
...         if self.index < len(self.elements):
...             return self.elements[self.index]
...         raise StopIteration
...     def __iter__(self):
...         return self

>>> iterable = Iterable("apple", "banana", "orange")
>>> iterator = iter(iterable)
>>> next(iterator)
'apple'
>>> next(iterator)
'banana'
>>> next(iterator)
'orange'
>>> next(iterator)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    next(iterator)
  File "<input>", line 9, in __next__
    raise StopIteration
StopIteration

To signal that all elements have been exhausted, an iterator must raise the StopIteration exception. However, it’s optional since your iterator might be infinite, in which case it will never exhaust iterable’s elements.

Note that iterators are typically one-use only, which means that once they reach the last element in the iterable, they’re no longer useful. You must create a brand new iterator instance, which will point the first element in an iterable.

Become a Member to join the conversation.