Unstructuredunstructured 0.27.6 · Python 3.11+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
21 small wins to finish your pathNext lesson

Reading a document with plain Python

Before the library, the problem. The shop's shipping rules live in a markdown file, and a program that wants to answer questions from it has to break it into sections first.

Here is the file. It is the first of the six documents this course works with.

markdown
# Shipping

## Delivery times

Standard delivery takes 3 working days. Express delivery arrives the next working day.

## Tracking

- Every order gets a tracking code by email.
- Codes go live 4 hours after dispatch.

## Missed deliveries

The courier tries twice. After that the parcel returns to our warehouse and we refund the order.

Splitting it by hand

Markdown marks a heading with hashes, so splitting on them is a few lines of ordinary Python.

Example
text = open("shipping.md").read()
for block in text.split("\n## "):
    first = block.strip().splitlines()[0]
    print(len(block), "|", first)

That works. Four blocks, each starting with its heading, each with a length. A program could search those blocks and tell a customer how long delivery takes.

It works because we read the file first and wrote a rule that fits it. The rule is split on a newline followed by two hashes, and it is true of this file and of nothing else.

What the rule does not know

The blocks are strings. Nothing in them says which one is a heading and which one is a list of two bullets. Nothing says the third block belongs under the first. Nothing says which file any of it came from, because we only opened one.

Example
text = open("shipping.md").read()
blocks = text.split("\n## ")
print(type(blocks[1]))
print(repr(blocks[1][:40]))

A string, and a string that still has its own heading stuck to the front of it. Every question you might later ask, which section is this, is it a heading, where did it come from, has to be answered by writing another rule.

Lesson 2 adds the second document, and the rule stops being true.

Try it yourself
  • Change the split to "\n# " and see how many blocks come back.
  • Print blocks[0]. Why does the first block look different from the rest?
  • Add a third hash level to the file and watch the splitter ignore it.

Little by little, you're building something great.