Three files, three parsers
Before Docling does it in one line, it is worth doing it the hard way. Three files, three pieces of code, three different shapes of answer.
Here are the first two files. Put them in a folder called handbook and work inside it.
# Shipping
## Delivery times
Standard delivery takes 3 working days. Express delivery arrives the next working day.
## Tracking
- Every order gets a tracking code by email.
- Codes go live 4 hours after dispatch.<h1>Refunds</h1>
<p>Refunds go back to the card used for the order.</p>
<h2>How long it takes</h2>
<table>
<tr><th>Payment method</th><th>Working days</th></tr>
<tr><td>Card</td><td>5</td></tr>
<tr><td>Bank transfer</td><td>8</td></tr>
</table>Headings out of Markdown
Markdown marks a heading with hashes at the start of a line, so a heading list is a filter.
headings = [line.strip() for line in open("shipping.md") if line.startswith("#")]
for heading in headings:
print(heading)That works, and it tells you nothing about how deep each heading sits unless you count hashes yourself. It also finds nothing at all in the next file, which says the same kind of thing with different characters.
Headings out of HTML
import re
html = open("refunds.html").read()
print(re.findall(r"<h(\d)>(.*?)</h\1>", html))A different rule for the same idea, and a regular expression over HTML is famously fragile: one attribute inside the tag and it stops matching. The table in that file needs a third rule again.
Rows out of CSV
order,status,courier
A17,shipped,Bluedart
B02,packing,
C41,delivered,Delhiveryimport csv
with open("orders.csv") as handle:
rows = list(csv.reader(handle))
print(rows[0])
print(len(rows) - 1, "orders")Three files and three parsers, and the three answers do not line up: a list of strings, a list of pairs, a list of lists. Anything that wants to search all three has to flatten them itself.
And then the PDF
The fourth file is a PDF. Opening it the same way shows what a PDF actually is.
raw = open("returns.pdf").read()
print(raw[:34])
start = raw.index("BT ")
print(raw[start:start + 90])Drawing instructions. Somewhere inside there is text, but it is stored as commands that put characters at coordinates, and nothing in the file says which of those characters were a heading. There is no fourth parser to write here in ten lines.
- Add a line to
shipping.mdthat starts with a hash inside a code fence, and watch the Markdown filter count it as a heading. - Change
<h2>inrefunds.htmlto<h2 id="x">and run the regular expression again. - Print
rows[2]from the CSV and note that the empty courier is an empty string, not a missing value.
Little by little, you're building something great.