Custom exceptions

A custom exception is an error class of your own that wraps the original failure with the file and the line it came from, so the one line that reaches a log file still says where to look.

Last updated: 19 Oct, 2026 · MLOps

The logging module gave the project a record. This lesson makes the worst lines in that record useful. A pipeline that fails writes one message into the log, and the default message for a failure deep inside a transform tells you what broke without telling you where.

Reading the error you get for free

Start with the failure as Python reports it. A split that comes back empty divides by zero:

def average(total, count):
    return total / count      # count is 0 when a split comes back empty

print(average(72, 0))

Traceback (most recent call last):
  File "main.py", line 4, in <module>
    print(average(72, 0))
  File "main.py", line 2, in average
    return total / count      # count is 0 when a split comes back empty
ZeroDivisionError: division by zero

• The type and the message are there: a ZeroDivisionError, division by zero. On a terminal that is enough.
• The traceback is what carries the location, and it is the part a log file does not get. A component that catches this and logs str(err) records the words division by zero and nothing about where they happened.
• In a pipeline of four stages that message fits any of them, which is the gap the next few lines close.

The clip calls the class CustomException; this project calls the same class SPException after its package, and the body is unchanged.

Reading the location off sys.exc_info

sys.exc_info returns three things about the exception being handled, and the third is the traceback object. The frame it points at knows its file and its line number, so those two facts can be folded into the message itself.

import sys


def error_detail(error, exc_info):
    """Build a message naming the file and line the error came from."""
    _, _, tb = exc_info
    file_name = tb.tb_frame.f_code.co_filename
    return f"Error in {file_name}, line {tb.tb_lineno}: {error}"

Wrapping it in an exception class

The class is four lines. It subclasses Exception, builds the detailed message and hands that to the parent, which is what makes str(err) and the last line of a traceback carry the location.

class SPException(Exception):
    def __init__(self, error, exc_info=None):
        # the live exception is read off sys when the caller does not pass one
        exc_info = exc_info or sys.exc_info()
        super().__init__(error_detail(error, exc_info))

Reading the same failure through SPException

Now the component catches the error and raises its own. The message is one line, and it names the file and the line:

from sp.exception import SPException

try:
    average(72, 0)
except Exception as err:
    print("logged:", SPException(err))

logged: Error in main.py, line 4: division by zero

• The message now begins with the file and the line, so the one string a log file keeps points at a place in the code.
• The original message is still inside it. Wrapping adds context and throws nothing away, which matters when the real cause is a missing column name.
• The line it names is where the call was made, not where the division happened. That is the frame sys.exc_info hands back first, and it is the frame a reader cares about: which call in the pipeline failed.
• exc_info defaults to sys.exc_info(), so a component inside an except block writes SPException(err) and passes no second argument.

How a component uses it

The pattern is the same in every stage: do the work, catch anything, log the wrapped error, then raise it so the pipeline stops rather than carrying on with no data.

from sp.exception import SPException
from sp.logger import logger

def ingest(url):
    try:
        return pd.read_csv(url)
    except Exception as err:
        # the log keeps the location, and the raise stops the pipeline here
        logger.error(str(SPException(err)))
        raise SPException(err) from err

A bare exception against a wrapped one

Where the wrapped error pays for itself

• A scheduled retrain that failed overnight, where the log line is the whole investigation and a location in it saves a rerun.
• A service returning a 500, where the handler logs one line per request and cannot print a traceback to the caller.
• A pipeline of several stages, where catching SPException at the top separates your own failures from a library's.

Related

• Previous: The logging module
• Next: Config files, not constants
• Reference: Errors and exceptions in the Python documentation