Methodologies
Evaluation methods that preserve the boundary between evidence and conclusion
A methodology does more than collect observations. It defines how an evaluation is constructed, what makes its observations comparable, which rule is applied, what result can be issued, and where that result stops.
IBC Labs develops methods for evaluating complex and probabilistic systems under conditions in which the complete producing composition may not be available to the evaluator.
The purpose is not to overcome the interaction boundary by assumption. It is to produce stronger and more disciplined determinations from the evidence that is actually available.
Featured Methodology
From Observation to Determination
A Methodology for Evaluating Al Systems Under Incomplete Observability
The methodology begins by declaring the property being evaluated rather than treating "the system" as an undifferentiated object.
| What property is being evaluated? | The property must be observable and defined clearly enough to support comparison or relational evaluation. |
|---|---|
| What is the declared target of the evaluation? | The method distinguishes multiple-target comparisons from evaluations of one target across related conditions. |
| What stimulus, conditions, and observation basis apply? | The evaluator declares what is held constant, what is intentionally varied, and how the observation set is established. |
| What variation exists within the observation basis? | Repeated observations may be required before a difference can be distinguished from ordinary variability. |
| What comparison rule or relation will be applied? | The rule must evaluate the same declared property across the relevant observation surfaces. |
| What does the resulting evidence support? | The determination is reported together with its conditions, coverage, boundaries, and unresolved residual. |
| Differential evaluation | Compares multiple declared targets with respect to a common property. Targets don’t need to share architecture or implementation, but they must produce commensurable observations. A single nonmatching result isn’t automatically divergence; divergence requires a distinguishable difference in empirically established observation profiles. Detected difference signals interpretation need, not correctness, fault, or preference. |
|---|---|
| Metamorphic evaluation | Examines one declared target across related source and follow-up cases. Evaluator declares a transformation, conditions, and relationship to be evaluated. Relation may be satisfied, not satisfied, inapplicable, or unresolved. Satisfaction doesn’t prove correctness; non-satisfaction doesn’t prove fault. Declared relation defines evaluation scope but isn’t a correctness oracle. |
| Behavioral evaluation | Still under methodology development. Formal construction will be published after review of measurement object, evaluation boundaries, and relation to differential and metamorphic families. Current status should remain visible — incomplete methods must not be presented as complete for product story clarity. |
Methodological boundaries
- Complete producing composition
- Causality from observed difference alone
- Correctness from cross-system agreement
- Equivalence from the absence of detected divergence
- Stability outside the declared observation basis
- Sufficiency for a downstream business, regulatory, or reliance decision
Its output is a bounded determination: what the available evidence supports under declared conditions, together with what remains unresolved.
