Unstructuredunstructured 0.27.6 · Python 3.11+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
21 small wins to finish your pathNext lesson

Tables, and text_as_html

A table is one element, however many rows it has. Its text is useless and its metadata is not.

The refund policy has a two column table in it. Here is what the Table element says.

Example
from unstructured.partition.auto import partition

table = partition("refunds.html")[3]
print(table.category)
print(table.text)

Every cell, in reading order, separated by single spaces. Nothing marks the end of a row and nothing marks the header. Card 5 Bank transfer 8 is the whole table and it is not something you can work with.

The shape is in the metadata

Example
from unstructured.partition.auto import partition

table = partition("refunds.html")[3]
print(table.metadata.text_as_html)

The rows are back. This is the field to hand to anything downstream that needs the table as a table, and it is the reason a table element is worth keeping whole rather than splitting.

Look closely at the first row. It was <th> in the source file and it is <td> here. The library keeps the grid and drops the distinction between a header cell and a data cell, so a program that wants the column names has to take the first row and hope.

A CSV is a table

Example
from unstructured.partition.auto import partition

elements = partition("orders.csv")
print(len(elements), elements[0].category)
print(elements[0].metadata.text_as_html)

One element for the whole file. The empty courier cell in the second row came back as an empty tag rather than being skipped, so the columns still line up.

An embedder given table.text sees one long sentence of cell values, which matches almost any question weakly and none of them well. Lesson 20 shows the chunker's option for keeping a table out of the ordinary text chunks.
Try it yourself
  • Add a row to refunds.html and check that text_as_html grows.
  • Feed text_as_html to pandas.read_html and see the table come back as a dataframe.
  • Remove the <table> tags from the HTML and see what the elements become instead.

Slow is fine. Stopping is the only problem.