Docling from the command line

Installing Docling also installs a command. For converting a folder once, it is the whole program you were about to write.

docling shipping.md --to md --output out

That converts one file and writes out/shipping.md. The snippets below run the same command through Python so this page can capture what it produced; on your machine you would type the line above.

import subprocess
import sys
from pathlib import Path

def docling(*arguments):
    subprocess.run([sys.executable, "-m", "docling.cli.main", *arguments], check=True)

docling("shipping.md", "--to", "md", "--output", "out")
print(Path("out/shipping.md").read_text()[:44])

The output file is named after the input, with an extension for the format asked for. Give it several inputs and it converts them all in one process, which is the command line version of lesson 18's advice about reusing the converter.

docling("shipping.md", "refunds.html", "orders.csv",
        "--to", "md", "--output", "many")
print(sorted(path.name for path in Path("many").iterdir()))

Chunks without writing any code

--to chunks runs the chunker from part 6 and writes one JSON object per chunk.

docling("shipping.md", "--to", "chunks", "--output", "cut")
first = Path("cut/shipping.chunks.jsonl").read_text().splitlines()[0]
print(first[:96])

Text, headings, the addresses it came from and a token count, ready for an indexer. --chunks-max-tokens and --chunks-tokenizer set the same things lesson 24 sets in Python.

One output format lies

The list of output formats includes text. Ask for it and look at what lands on disk.

docling("refunds.html", "--to", "text", "--output", "plain")
print(Path("plain/refunds.txt").read_text())

A file called .txt containing Markdown, hashes and table pipes and all. The command asks for plain text using the deprecated argument from lesson 10, and because that argument is now ignored the Markdown comes through untouched. export_to_text in Python still does the right thing; the command line does not.