Collections or legacy: what changed in 0.4

Two imports in this course look almost the same and behave differently. This lesson is the map, because every tutorial you find online is written against the older one.

Example

import warnings

with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    from ragas.metrics import Faithfulness as LegacyFaithfulness
    print(len(caught), "warning")
    print(str(caught[0].message)[:180])

The library tells you itself. The classes in ragas.metrics are the old API; the same names in ragas.metrics.collections are the new one.

	ragas.metrics (legacy)	ragas.metrics.collections (current)
How you score	`await metric.single_turn_ascore(sample)`	`metric.score(user_input=..., response=...)`
What you pass	A `SingleTurnSample`	Plain keyword arguments
The judge	`BaseRagasLLM`, returns text	`InstructorBaseRagasLLM`, returns objects
Works with `evaluate()`	Yes	No, it type-checks the old base class
Future	Removed in 1.0	The one to write new code against

So the choice is not about taste. If you want evaluate() and its pandas table, you are using legacy metrics and a legacy judge. If you want the current API, you score metrics yourself and run them inside an experiment, which is what part 4 built.

Pin your versions

This library moves fast enough that a tutorial from last year may not run. This course pins ragas==0.4.3 and langchain-community<0.4, because 0.4.3 imports a class that langchain-community removed.

Try it yourself

Import both Faithfulness classes in one file and print their modules.
Try passing a collections metric to evaluate() and read the error.

PreviousEvaluationDataset and evaluate(): the classic path

Little by little, you're building something great.