Writing Custom Iterators in Lua

Iterators let you step through data structures one element at a time. Lua’s generic for loop has a built-in protocol for them, and once you understand that protocol, you can build iterators for anything: trees, graphs, file formats, API responses.

How the Generic for Loop Works

The generic for loop calls your iterator on every iteration. It passes two values each time: the invariant state (the collection, which never changes) and the control variable (the current position).

for <var1>, <var2>, ... in <iter>, <state>, <init> do
  -- loop body
end

On the first iteration, Lua calls <iter>(<state>, <init>). On subsequent iterations, it calls <iter>(<state>, <var1>) — the first returned value from the previous call becomes the new control variable.

The loop stops when the iterator returns nil. If it returns multiple values, the first becomes the next control variable and the rest are assigned to the loop variables.

This is the protocol. Everything in this guide builds on it.

Built-in Iterators: pairs and ipairs

Before writing your own, understand what Lua gives you.

ipairs

Iterates integer-keyed tables sequentially from index 1. Stops at the first nil.

for i, v in ipairs({10, 20, 30}) do
  print(i, v)  -- 1 10, 2 20, 3 30
end

pairs

Iterates all key-value pairs. Order is not guaranteed for non-integer keys.

for k, v in pairs({a = 1, b = 2, [3] = 3}) do
  print(k, v)
end

pairs is implemented in C and uses the next function internally. You can call next directly:

for k, v in next, t, nil do
  print(k, v)
end

This is equivalent to pairs(t).

Stateless Iterator with a Factory Function

A stateless iterator only uses its two arguments (state and control). The factory function returns three values: the iterator function, the invariant state, and the initial control value.

local function squares(max)
  local function iter(state, i)
    i = i + 1
    if i <= state.max then
      return i, i * i
    end
  end
  return iter, {max = max}, 0
end

for i, sq in squares(5) do
  print(i, sq)  -- 1 1, 2 4, 3 9, 4 16, 5 25
end

The iterator function iter is pure — it doesn’t store any state itself. All the state lives in the table {max = max} passed as the invariant state. The control variable i tracks position across calls.

This pattern works well when the traversal logic is straightforward. For more complex state, it gets awkward fast.

Closure-Based Stateful Iterator

If stateless iteration feels forced, use a closure. The iterator function carries state in its upvalues, so you don’t need to pass everything through the protocol.

local function list_iter(t)
  local i = 0
  local n = #t
  return function()
    i = i + 1
    if i <= n then
      return t[i]
    end
  end
end

for v in list_iter({"one", "two", "three"}) do
  print(v)  -- one, two, three
end

Here list_iter returns a closure that reads i, n, and t from its upvalues. The generic for loop just keeps calling that function until it returns nil.

The tradeoff: this iterator can’t be paused and resumed from the same position because the state lives inside the closure, not in the protocol arguments.

Coroutine-Based Iterator

Coroutines shine when the iteration logic is complex or naturally recursive. Each coroutine.yield produces a value for the loop.

local function permutations(t)
  return coroutine.wrap(function()
    local function permgen(a, n)
      if n == 0 then
        coroutine.yield(a)
      else
        for i = 1, n do
          a[n], a[i] = a[i], a[n]
          permgen(a, n - 1)
          a[n], a[i] = a[i], a[n]  -- backtrack
        end
      end
    end
    local a = {}
    for _, v in ipairs(t) do table.insert(a, v) end
    permgen(a, #a)
  end)
end

for perm in permutations({"A", "B", "C"}) do
  print(table.concat(perm, ","))  -- prints all 6 permutations
end

coroutine.wrap returns an iterator function. When that function calls coroutine.yield(value), the value bubbles up to the for loop’s loop variable. When the coroutine finishes, coroutine.wrap returns nil and the loop ends.

This is cleaner than trying to cram a recursive traversal into a stateless iterator. You write the logic naturally, and yield handles the communication.

Iterator with Complex State

When an iterator needs multiple pieces of state, pack them into a table used as the invariant state.

local function words(s)
  local state = {s = s, pos = 1}
  return function(st)
    local start = st.s:find("%S", st.pos)
    if not start then return nil end
    local finish = st.s:find("%s", start + 1)
    local word
    if finish then
      word = st.s:sub(start, finish - 1)
      st.pos = finish + 1
    else
      word = st.s:sub(start)
      st.pos = #st.s + 1
    end
    return word
  end, state
end

for w in words("hello world from lua") do
  print(w)  -- hello, world, from, lua
end

The state table holds both the string and the current position. The iterator function reads and writes these fields on every call.

Common Mistakes

Forgetting that nil ends the loop

The loop only stops when the iterator returns exactly nil. Returning false, 0, or an empty table does not end it.

for v in function() return false end do  -- infinite loop
  print("oops")
end

Always return nil to signal done.

ipairs stops at nil

If your array has gaps, ipairs stops at the first nil:

local t = {1, nil, 3}
for i, v in ipairs(t) do print(i, v) end  -- only prints 1 1

Use pairs for sparse arrays.

Modifying the table you’re iterating over

Removing keys during pairs iteration can cause skipped or duplicate elements:

for k, v in pairs(t) do
  t[k] = nil  -- may skip some keys
end

Copy the keys first if you need to delete while iterating.

Confusing numeric for with generic for

Numeric for (for i = 1, 10) and generic for (for k, v in pairs(t)) are separate constructs. Numeric for manages its own counter. Generic for calls an iterator function. Don’t mix them up.

When to Use Each Pattern

Use a stateless iterator when the traversal is simple and sequential. The factory pattern keeps things clean and lets you reuse iterator functions.

Use a closure iterator when you want to hide state from the protocol entirely. It’s easier to write but can’t be reset mid-iteration.

Use a coroutine when the traversal is complex, recursive, or involves branching. You get natural control flow at the cost of some overhead.

Conclusion

Lua’s iterator protocol is simple: an iterator function, an invariant state, and a control variable. Once that clicks, you can build iterators for any data structure. Start with the stateless pattern for sequential traversals, reach for coroutines when the logic gets tangled, and use closures for quick cases where reusability doesn’t matter.

See Also: Lua Metatables | Coroutine Basics | Tables in Lua