Metadata: what gets embedded and what the model sees

Each node carries metadata into two places: the text that is embedded, and the text the model reads. LlamaIndex lets you choose, per key, which it goes into.

from llama_index.core import Document
from llama_index.core.schema import MetadataMode

doc = Document(text="Standard delivery takes 3 to 5 working days.", metadata={"file_name": "delivery.md", "reviewed_by": "ops-team"})
print(doc.get_content(metadata_mode=MetadataMode.EMBED))
print("---")
print(doc.get_content(metadata_mode=MetadataMode.LLM))

get_content with MetadataMode.EMBED shows what the embedding model receives, and LLM what the model reads. For a Document you create yourself, both include every metadata key, written as key: value lines above the text. SimpleDirectoryReader sets exclusions for the file keys it adds, which is why lesson 9's prompt shows none.

Choosing per key

doc.excluded_embed_metadata_keys = ["reviewed_by"]
doc.excluded_llm_metadata_keys = ["reviewed_by"]
print(doc.get_content(metadata_mode=MetadataMode.EMBED))
print("---")
print(doc.get_content(metadata_mode=MetadataMode.LLM))

reviewed_by is useful to keep on the node for filtering or auditing, and useless for matching a customer's question or answering it, so it is excluded from both. file_name stays in the model's view here, so a model could name the file it answered from.

Rule of thumb: embed what helps a question find the chunk, such as a title or product name; show the model what helps it answer or cite; keep everything else out of both.