Skip to content

test: preprocessing._transformers — CRMCleaner/FiscalYearTransformer defensive branches are untested (87% coverage) #62

Description

@shivamlalakiya

Summary

philanthropy/preprocessing/_transformers.py sits at 87% branch coverage —
the lowest of any file in the risk tier that make riskcov enforces at 93%.
The gap is a cluster of defensive branches in CRMCleaner and
FiscalYearTransformer that no test reaches:

  1. _get_pandas_output's two fallback paths. The function first tries the
    modern sklearn.utils._set_output._get_output_config path, falling back to
    a manual _sklearn_output_config attribute check if that import or call
    fails, and within that fallback there's a further branch between a plain
    "pandas" string and a {"dense": "pandas"} dict shape. No test forces the
    ImportError/AttributeError path or exercises the dict-shaped config.
  2. The except Exception re-raise / retry branches in fit and transform
    on both CRMCleaner and FiscalYearTransformer.
    Each of these four
    methods wraps validate_data(...) in a try/except Exception, re-raising
    only when the message contains "Complex data not supported" and
    otherwise retrying with X.astype(object). Existing tests only exercise
    the "no exception" and "complex data raises" cases — the retry-succeeds
    path (some other validate_data exception, e.g. a mixed-type object array
    that a plain dtype check trips on but .astype(object) fixes) is never
    taken.
  3. The plain ValueError("Complex data not supported") raise inside
    CRMCleaner.fit/transform and FiscalYearTransformer.fit/transform
    when validation succeeds but the resulting array is complex-typed
    — as
    opposed to the case where validate_data itself raises on complex input.
    Confirm with coverage which of the two complex-data raise sites in each
    method is the untested one before writing the test.

Where

File Locate with
philanthropy/preprocessing/_transformers.py grep -n "_get_pandas_output|except Exception|Complex data not supported" philanthropy/preprocessing/_transformers.py

What to change

No source change — this is test-only. Run coverage first to see exactly which
lines are still red before writing tests, since some of the branches above may
shift as other issues in this file land:

python -m pytest tests/test_preprocessing.py --cov=philanthropy.preprocessing._transformers --cov-report=term-missing -q
  1. For _get_pandas_output: add a test that monkeypatches
    sklearn.utils._set_output._get_output_config to raise AttributeError
    (or removes it) and sets estimator._sklearn_output_config = {"transform": {"dense": "pandas"}} directly, asserting the dict-shaped fallback still
    returns True.
  2. For the retry-succeeds branch: find or construct an input that
    validate_data rejects on its first pass but accepts once cast to
    dtype=object (a CRMCleaner/FiscalYearTransformer sibling test already
    exists for the complex-data case — locate with
    grep -n "Complex data" tests/test_preprocessing.py — use it as the
    pattern for constructing tricky input, but pick a case that recovers
    instead of raising).
  3. For the complex-result-after-successful-validation raise: construct X
    that validate_data accepts (e.g. object dtype) but whose array is
    np.iscomplexobj(...) true after validation.

Tests to add or extend

  • File: tests/test_preprocessing.py
  • Add: test_get_pandas_output_dict_shaped_config_fallback
  • Add one retry-succeeds test per class (CRMCleaner, FiscalYearTransformer)
    for fit and/or transform, named to match the existing
    test_crm_cleaner_* / test_fiscal_year_transformer_* conventions in that
    file (locate with grep -n "class Test" tests/test_preprocessing.py).

Done when

python -m pytest tests/test_preprocessing.py --cov=philanthropy.preprocessing._transformers --cov-report=term-missing -q

philanthropy/preprocessing/_transformers.py reads at or above the 93%
risk-tier floor with no branches from the three groups above left in
Missing.

make riskcov

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions