PDF and images: the extra this course skips

PDF is the format everyone comes to this library for, and it is the one this course does not install. This lesson says what happens if you try, what the install would cost, and what the three PDF strategies mean.

Start with the failure, because it is a clear one.

from unstructured.partition.auto import partition

try:
    partition("policy.pdf")
except ImportError as error:
    print(error)

The dispatcher got as far as choosing partition_pdf and stopped. It never opened the file, which is why a file containing nonsense produced the same message a real PDF would. The library checks the packages before it does any work.

What the extra costs

pip install "unstructured[pdf]" is an alias for the image extra, and that one pulls unstructured-inference, which requires torch, transformers, accelerate, timm, onnx, onnxruntime, opencv and scipy. About three gigabytes. Two of the strategies below also need programs installed outside pip: poppler for turning pages into images, and tesseract for reading them.

This course leaves it out so that pip install is the whole setup and every lesson runs in under a second. If PDF is your job, install the extra; nothing else in this course changes.

The three strategies

Once installed, partition_pdf takes a strategy argument, and choosing it is the whole skill. The three names mean three different amounts of work.

• fast pulls the text the PDF already contains, with pdfminer. It is the quickest by a wide margin and it returns nothing at all for a scanned page, because a scanned page contains an image and no text.
• hi_res renders each page, runs a layout model over it to find the titles, paragraphs and tables, and reads the text from the model's boxes. It is the only one that gets table structure right, and it is slow.
• ocr_only renders each page and runs tesseract over it, with no layout model. For a scan with no tables it gets the words without the download.

The default is auto, which picks between them by looking at the document and at what you asked for. If you asked for the text of a PDF that has text, you get fast.