Partitioning a stream instead of a path

Documents do not always arrive as files on disk. They arrive as uploads, as rows in a database, as the body of a request. partition takes those too, with one trap on the way in.

The obvious guess is wrong, so start there.

from unstructured.partition.auto import partition

try:
    partition(text="Refunds take five working days.")
except ValueError as error:
    print(error)

partition has no text parameter. The per-format functions do, and the documentation shows them constantly, so the guess is a natural one. The dispatcher needs something it can sniff or a name it can read, and a bare string is neither.

A file object

from unstructured.partition.auto import partition

with open("shipping.md", "rb") as handle:
    elements = partition(file=handle)
print(len(elements), elements[0].metadata.filename)

Eight elements, the same as lesson 3, and a filename of None. The stream had to be opened in binary mode, and the type was worked out from the bytes rather than from a name.

The missing filename matters more than it looks. It is in the metadata that a citation reads, and lesson 9 showed it is part of every element id. So when the name is known, pass it.

from unstructured.partition.auto import partition

with open("shipping.md", "rb") as handle:
    elements = partition(file=handle, metadata_filename="shipping.md")
print(elements[0].metadata.filename, elements[0].id[:8])

metadata_filename only labels the elements. It does not open anything, and it does not have to be a path that exists, which is exactly what an upload needs.

Going straight to one partitioner

When you already know the format, call its function directly. Those are the ones that take text.

from unstructured.partition.text import partition_text

for element in partition_text(text="Refunds take five working days."):
    print(element.category, "|", element.text)

Every format has one: partition_html, partition_docx, partition_csv and so on, twenty-three of them. Calling one skips the detection in lesson 10 entirely, which is faster and is the right choice when the format is fixed.