Unstructuredunstructured 0.27.6 · Python 3.11+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
21 small wins to finish your pathNext lesson

Looking at what came back

Before doing anything with the elements, learn to look at them. Every problem in the rest of this course is diagnosed by printing the element list and reading it.

An element has three things worth printing early: what it is, what it says, and everything else. The first two are attributes.

Example
from unstructured.partition.auto import partition

element = partition("shipping.md")[2]
print(element.category)
print(element.text)

category is the label as a string, and it is the same word as the class name. text is the words with the formatting taken off. Neither of them is the whole element.

A function you will use in every lesson

Printing the index, the label and the start of the text is enough to understand almost any result, so it is worth a function.

Example
from unstructured.partition.auto import partition

def show(elements):
    for i, element in enumerate(elements):
        print(f"{i:2d} {element.category:14s} {element.text[:48]!r}")

show(partition("refunds.html"))

The index matters more than it looks. When a later lesson says the table came back wrong, the way you find out is by printing the list and pointing at a row number.

Everything else

The rest of an element comes out with to_dict, which is also what gets written when the elements are saved to a file.

Example
from unstructured.partition.auto import partition

element = partition("shipping.md")[2]
for key, value in element.to_dict().items():
    print(key, "=", value)

Four keys. The type and the text you have already seen. element_id is a name for this piece, and lesson 9 is about where it comes from. metadata is everything the library knows about where the piece was found, and that is lesson 6.

Run this and you will probably see a line on the terminal saying libmagic is unavailable but assists in filetype detection. It is a warning, not an error, and lesson 10 explains what it changes.
Try it yourself
  • Change show to print element.text in full and run it on orders.csv.
  • Print element.to_dict()["metadata"]["filetype"] for each of the three files.
  • Call show on an empty list and check it prints nothing rather than failing.

This is what real progress feels like.