Tables, and text_as_html
A table is one element, however many rows it has. Its text is useless and its metadata is not.
The refund policy has a two column table in it. Here is what the Table element says.
from unstructured.partition.auto import partition
table = partition("refunds.html")[3]
print(table.category)
print(table.text)Every cell, in reading order, separated by single spaces. Nothing marks the end of a row and nothing marks the header. Card 5 Bank transfer 8 is the whole table and it is not something you can work with.
The shape is in the metadata
from unstructured.partition.auto import partition
table = partition("refunds.html")[3]
print(table.metadata.text_as_html)The rows are back. This is the field to hand to anything downstream that needs the table as a table, and it is the reason a table element is worth keeping whole rather than splitting.
Look closely at the first row. It was <th> in the source file and it is <td> here. The library keeps the grid and drops the distinction between a header cell and a data cell, so a program that wants the column names has to take the first row and hope.
A CSV is a table
from unstructured.partition.auto import partition
elements = partition("orders.csv")
print(len(elements), elements[0].category)
print(elements[0].metadata.text_as_html)One element for the whole file. The empty courier cell in the second row came back as an empty tag rather than being skipped, so the columns still line up.
table.text sees one long sentence of cell values, which matches almost any question weakly and none of them well. Lesson 20 shows the chunker's option for keeping a table out of the ordinary text chunks.- Add a row to
refunds.htmland check thattext_as_htmlgrows. - Feed
text_as_htmltopandas.read_htmland see the table come back as a dataframe. - Remove the
<table>tags from the HTML and see what the elements become instead.
Slow is fine. Stopping is the only problem.