What this install can read, and what it cannot
The install in lesson 3 named seven extras. Rather than trusting the list, ask the library.
Every file type carries the Python packages it needs and the name of the extra that installs them. Both are attributes, and the library has a function for asking whether a package is importable.
from unstructured.file_utils.model import FileType
from unstructured.utils import dependency_exists
pdf = FileType.PDF
print(pdf.extra_name, pdf.importable_package_dependencies)
print(dependency_exists("pdfminer"), dependency_exists("docx"))PDF names three packages and asks for the pdf extra. pdfminer, the one the check below reports on, is not installed. Word needs docx, which is, because docx was one of the seven extras.
The whole table
from unstructured.file_utils.model import FileType
from unstructured.utils import dependency_exists
for kind in FileType:
if not kind.is_partitionable or kind.extra_name == "audio":
continue
ready = all(dependency_exists(d)
for d in kind.importable_package_dependencies)
print(f"{kind.name:6s} {kind.extra_name or 'none':8s}"
f" {'yes' if ready else 'no'}")Sixteen formats this install can read and eleven it cannot, and the middle column names the extra that would fix each one. Formats with none in the middle need no extra at all: HTML, plain text, XML, JSON, NDJSON and email are handled by the base package.
The eleven missing ones fall into two groups. epub, odt, org, rst and rtf all want pypandoc, which needs the pandoc program installed outside pip. The other six are PDF and the five image formats, which are lesson 12.
What the table cannot see
It is a check on Python packages and nothing else. DOC and PPT, the formats from before Office moved to zip files, say yes here and still fail at run time: the library converts them by running LibreOffice from the command line, and a missing soffice is not a missing Python package.
unstructured[all-docs]. The all-docs install includes the image extra, which pulls torch, transformers and onnxruntime and takes the install from about two hundred megabytes to about three gigabytes.- Count how many formats in the table need no extra at all.
- Run
dependency_exists("unstructured_inference")and connect it to the PDF row. - Look up
FileType.XLSX.importable_package_dependenciesand check both packages are installed.
Little by little, you're building something great.