How partition picks a partitioner

partition is a dispatcher. It works out what the file is, then hands it to the function for that format. Both halves are worth seeing, because when a file comes back wrong it is almost always the first half.

The decision is its own function and you can call it.

from unstructured.file_utils.filetype import detect_filetype

for name in ["shipping.md", "refunds.html", "orders.csv", "ravi.eml"]:
    print(name, "->", detect_filetype(name))

A FileType member for each, and each member knows the function that handles it. That is all the dispatch is.

Four questions, in order

The detector asks four things and takes the first answer it gets.

When the extension lies

Here is a web page that somebody saved with the wrong extension.

<h1>Refunds</h1><p>Refunds take five working days.</p>

from unstructured.partition.auto import partition

for element in partition("policy.txt"):
    print(element.category, "|", element.text)

One element, and its text is the file with the tags still in it. Nothing raised, nothing warned. The fourth question answered txt and the text partitioner did exactly what it was asked to do with a line of HTML.

This is the failure worth recognising, because it looks like a bug in the HTML parser and it is not. The fix is the second question.

from unstructured.partition.auto import partition

for element in partition("policy.txt", content_type="text/html"):
    print(element.category, "|", element.text)

Two elements, correctly labelled. content_type jumps the queue, and it is the one argument worth reaching for whenever a file comes back as a single suspicious blob.