parent_id: the document's own outline

The elements come back as a flat list, but a document is not flat. Two metadata fields keep the shape it had.

category_depth is how deep a heading sits, counting from zero. parent_id is the id of the heading an element belongs under.

from unstructured.partition.auto import partition

for element in partition("shipping.md"):
    print(f"{element.category:14s} id={element.id[:8]}"
          f" depth={element.metadata.category_depth}"
          f" parent={str(element.metadata.parent_id)[:8]}")

The first heading has depth zero and no parent, because it is the top of the file. The three ## headings have depth one and all name the same parent, which is the first heading. The paragraphs have no depth at all, because a paragraph is not a heading, and each one names the heading above it.

Walking back up

Because the parent is an id rather than a position, finding the section an element belongs to is a dictionary lookup.

from unstructured.partition.auto import partition

elements = partition("shipping.md")
by_id = {element.id: element for element in elements}

for element in elements:
    parent = by_id.get(element.metadata.parent_id)
    if parent:
        print(f"{parent.text:20s} <- {element.text[:34]}")

Every paragraph and every bullet now knows its section by name. That is the line the handbook in lesson 27 uses to say this answer came from the Tracking section of shipping.md.

What fills it in

Only formats that have real headings can fill these in. Markdown, HTML and Word all do. A CSV file has no headings, so its one table has neither field.

from unstructured.partition.auto import partition

element = partition("orders.csv")[0]
print(element.metadata.category_depth, element.metadata.parent_id)

Two Nones, which is the honest answer. A pipeline that assumes every element has a parent will fail on the first spreadsheet it meets.