StatisticalAnalysisHelper Module¶

Notes

The additive term b does not affect covariance but is kept for API symmetry with affine mean.

AffineMean(data, A, b, isMean=True)[source]¶

Compute the mean after an affine transform.

Implements E[A*X + b] = A * E[X] + b. If isMean=False the method will compute the mean from raw samples first.

\[\mathbb{E}[A X + b] = A \; \mathbb{E}[X] + b\]

Parameters:

data (numpy.ndarray) – Either a mean vector (1-D) or raw samples (2-D) if isMean=False.
A (numpy.ndarray) – Linear transform matrix.
b (numpy.ndarray or scalar) – Additive bias.
isMean (bool) – If True data is treated as a mean vector; if False the mean will be computed via RowsMean.

Returns:

Transformed mean vector.

Return type:

Area(data)[source]¶

Compute the area (zeroth moment) of an image or 2D array.

Parameters:: data (numpy.ndarray) – 2D array representing an image or spatial distribution.
Returns:: Zeroth spatial moment (sum over pixels) which corresponds to area.
Return type:: float

References

skimage.measure.moments

Centroid(data)[source]¶

Compute the centroid coordinates of a 2D image/array using spatial moments.

Parameters:: data (numpy.ndarray) – 2D array representing an image or spatial distribution.
Returns:: (x_centroid, y_centroid) as floats.
Return type:: tuple

Notes

Uses raw (not central) moments so typical centroid formula applies: (M10/M00, M01/M00).

ChiSquared(X, y, withCorrection=False)[source]¶

Compute the Pearson chi-squared statistic from two categorical arrays.

Parameters:

X (array-like) – Categorical observations for variable X.
y (array-like) – Categorical observations for variable Y.
withCorrection (bool) – If True apply Yates’ continuity correction.

Returns:

Chi-squared statistic.

Return type:

Notes

This function returns the test statistic only (no p-value).

ColumnsMean(data)[source]¶

Compute the mean for each column of a 2D array.

Parameters:: data (numpy.ndarray) – 2D array with shape (rows, cols).
Returns:: 1-D array with mean computed along axis=1.
Return type:: numpy.ndarray

Count(data)[source]¶

Return the total number of elements in the input.

Parameters:: data (array-like) – Input array or sequence.
Returns:: Number of elements.
Return type:: int

CountDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Count non-masked elements using a dynamic wrapper.

This delegates to Dynamic(np.ma.count, …), allowing axis and aggregation control.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return the mean of the counts.
perLastAxis (bool) – If True count along the last axis.
axis (int, optional) – Axis to count along.
keepdims (bool) – Whether to keep dimension for reduction.

Returns:

Count or aggregated count depending on flags.

Return type:

int or numpy.ndarray

CovarianceMatrix(data)[source]¶

Compute the sample covariance matrix of row-wise observations.

\[\mathrm{Cov}(X) = \frac{(X - \mu)^{T} (X - \mu)}{N},\quad \mu = \mathrm{RowsMean}(X)\]

Parameters:: data (numpy.ndarray) – 2-D array with samples along axis 0 (shape: N x D).
Returns:: Covariance matrix (D x D) computed as (X - mean).T @ (X - mean) / N.
Return type:: numpy.ndarray

CumulativeDistributionFunction(data, bins=10, range=None)[source]¶

Compute a discrete cumulative distribution function by binning data.

\[\mathrm{CDF}(k) = \sum_{i=1}^{k} p_i \quad\text{where } p_i = \frac{\text{hist}_i}{\sum_j \text{hist}_j}\]

Parameters:

data (array-like) – Input samples.
bins (int) – Number of histogram bins.
range (tuple or None) – Range for histogram bins.

Returns:

CDF values for the histogram bins.

Return type:

CumulativeFrequency(data, bins=10, returnFreqOnly=True, returnMean=False)[source]¶

Wrapper around scipy.stats.cumfreq to compute cumulative frequencies.

Parameters:

data (array-like) – Input samples.
bins (int) – Number of bins to use for cumulative frequency.
returnFreqOnly (bool) – If True return only the frequency array.
returnMean (bool) – If True return the mean of returned arrays instead of arrays.

Returns:

Depending on flags returns frequency array or full cumfreq output.

Return type:

array or tuple

DescriptiveStatistics(data, returnMean=False)[source]¶

Return a tuple of descriptive statistics using scipy.stats.describe.

Parameters:

data (array-like) – Input data sample.
returnMean (bool) – If True aggregate vector results into means.

Returns:

(nobs, mean, variance, kurtosis) where elements may be scalars or arrays depending on input.

Return type:

DispersionRatio(X)[source]¶

Compute the dispersion ratio between arithmetic and geometric means.

\[\mathrm{AM} = \frac{1}{N} \sum_{i=1}^{N} x_i ,\qquad \mathrm{GM} = \left(\prod_{i=1}^{N} x_i\right)^{1/N},\qquad \mathrm{DispersionRatio} = \frac{\mathrm{AM}}{\mathrm{GM}}\]

Parameters:: X (numpy.ndarray) – Input 2D array with samples along axis 0.
Returns:: Ratio of arithmetic mean to geometric mean for each column.
Return type:: numpy.ndarray

Dynamic(func, data, returnMean=False, perLastAxis=False, axis=None, keepdims=False, **kwargs)[source]¶

Generic dynamic wrapper to apply a reduction function along configurable axes.

Parameters:

func (callable) – Reduction function (e.g., np.mean, np.sum).
data (array-like) – Input array.
returnMean (bool) – If True return the mean of the reduced result.
perLastAxis (bool) – If True apply reduction along the last axis.
axis (int or None) – Axis to apply the reduction.
keepdims (bool) – Whether to keep reduced dimensions.
**kwargs – Additional keyword args forwarded to func.

Returns:

The reduced result (optionally averaged).

Return type:

scalar or numpy.ndarray

EmpiricalCumulativeDistributionFunction(data, returnMean=False)[source]¶

Compute the empirical cumulative distribution function (ECDF) values for data.

Parameters:

data (array-like) – Input samples.
returnMean (bool) – If True return the mean of ECDF values.

Returns:

ECDF values array or its mean.

Return type:

numpy.ndarray or float

Entropy(data)[source]¶

Compute Shannon entropy using scipy.stats.entropy averaged over provided distribution(s).

Parameters:: data (array-like) – Input probabilities or counts.
Returns:: Mean entropy value.
Return type:: float

HistEntropy(pdf)[source]¶

Compute entropy from a discrete probability mass function (base 2).

Parameters:: pdf (array-like) – Discrete probability distribution (must sum to 1).
Returns:: Entropy in bits.
Return type:: float

HistEnergy(pdf)[source]¶

Compute the energy of a discrete distribution (sum of squared probabilities).

Parameters:: pdf (array-like) – Probability distribution.
Returns:: Energy value (sum of squares).
Return type:: float

FValueUsingOneWayANOVA(X, y)[source]¶

Compute the one-way ANOVA F statistic for groups defined by y.

\[F = \frac{\mathrm{MSC}}{\mathrm{MSE}},\quad \mathrm{MSC} = \frac{\mathrm{SSC}}{df_1},\quad \mathrm{MSE} = \frac{\mathrm{SSE}}{df_2}\]

Parameters:

X (array-like) – Numeric observations (1-D or 2-D flattened to 1-D).
y (array-like) – Group labels for each observation.

Returns:

F-statistic computed from between- and within-group variances.

Return type:

Histogram(data, bins=10, range=None, returnMean=False)[source]¶

Compute a histogram using numpy and optionally return averaged results.

Parameters:

data (array-like) – Input data.
bins (int) – Number of bins.
range (tuple) – Range for histogram.
returnMean (bool) – If True return means of histogram counts and bin edges.

Returns:

(hist, binEdges) or their means if returnMean=True.

Return type:

InterquartileRange(X, axis=None)[source]¶

Compute the interquartile range (IQR) along the specified axis.

Parameters:

X (array-like) – Input data.
axis (int or None) – Axis along which to compute percentiles.

Returns:

IQR value(s).

Return type:

numpy.ndarray or scalar

KurtosisDynamic(data, ddof=0, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Compute kurtosis using a dynamic (axis-flexible) implementation.

Parameters:

data (array-like) – Input samples.
ddof (int) – Delta degrees of freedom for variance.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Kurtosis values.

Return type:

numpy.ndarray or scalar

Max(data)[source]¶

Return the maximum value in the input.

Parameters:: data (array-like) – Input data.
Returns:: Maximum value.
Return type:: scalar

MaxDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper for np.max with axis control.

Parameters and returns mirror Dynamic’s contract.

Mean(data, axis=None)[source]¶

Compute the arithmetic mean along the specified axis.

Parameters:

data (array-like) – Input array.
axis (int or None) – Axis to compute the mean over.

Returns:

Mean value(s).

Return type:

numpy.ndarray or scalar

HistMean(pdf, range)[source]¶

Compute the mean of a discrete histogram (pdf weighted by bin centers).

Parameters:

pdf (array-like) – Probability per bin.
range (array-like) – Bin center locations or range values.

Returns:

Histogram mean.

Return type:

MeanAbsoluteDifference(X, axis=0)[source]¶

Compute the mean absolute deviation from the mean along the given axis.

Parameters:

X (array-like) – Input data.
axis (int) – Axis along which to compute MAD.

Returns:

Mean absolute deviation.

Return type:

numpy.ndarray or scalar

MeanAbsoluteDifferenceDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic variant of mean absolute difference.

Mirrors MeanDynamic signature.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

MAD values.

Return type:

numpy.ndarray or scalar

RobustMeanAbsoluteDifference(X, axis=0, keepdims=False)[source]¶

Compute median-based mean absolute deviation (robust MAD).

Parameters:

X (array-like) – Input array.
axis (int) – Axis along which to compute the measure.
keepdims (bool) – Keep reduced dimensions.

Returns:

Robust MAD.

Return type:

numpy.ndarray or scalar

RobustMeanAbsoluteDifferenceDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper for the robust MAD function.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Robust MAD values.

Return type:

numpy.ndarray or scalar

MeanDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper for numpy mean with axis control.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Mean values.

Return type:

numpy.ndarray or scalar

Median(X, axis=None)[source]¶

Compute the median along an axis.

Parameters:

X (array-like) – Input data.
axis (int or None) – Axis along which to compute the median.

Returns:

Median value(s).

Return type:

scalar or array

MedianAbsoluteDeviation(data)[source]¶

Compute the median absolute deviation (MAD) for 1-D data.

Parameters:: data (array-like) – Input 1-D array.
Returns:: Median absolute deviation.
Return type:: float

MedianAbsoluteDeviationDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper for MAD.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

MAD values.

Return type:

numpy.ndarray or scalar

MedianDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper for numpy.median.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Median values.

Return type:

numpy.ndarray or scalar

RootMeanSquare(data, axis=None, keepdims=False)[source]¶

Compute the root mean square along an axis.

\[\mathrm{RMS} = \sqrt{\frac{1}{N}\sum_{i=1}^{N} x_i^2}\]

Parameters:

data (array-like) – Input data.
axis (int or None) – Axis over which to compute RMS.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

RMS values.

Return type:

numpy.ndarray or scalar

RootMeanSquareDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper for RMS computation.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

RMS values.

Return type:

numpy.ndarray or scalar

Min(data)[source]¶

Return the minimum value from the input.

Parameters:: data (array-like) – Input data.
Returns:: Minimum value.
Return type:: scalar

MinDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper for np.min.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Minimum values.

Return type:

numpy.ndarray or scalar

Mode(X)[source]¶

Compute the mode of integer-valued data using bincount.

Parameters:: X (array-like) – 1-D integer-valued data.
Returns:: The mode (most frequent value).
Return type:: int

Percentile(X, p, axis=0)[source]¶

Compute the p-th percentile of data.

Parameters:

X (array-like) – Input data.
p (float) – Percentile to compute (0-100).
axis (int) – Axis along which to compute percentile.

Returns:

Percentile value(s).

Return type:

scalar or array

Percentiles(data, ranges=[10, 20, 30, 40, 50, 60, 70, 80, 90, 100])[source]¶

Convenience to compute multiple percentiles at once.

Parameters:

data (array-like) – Input data.
ranges (list) – List of percentiles to compute.

Returns:

Percentile values corresponding to ranges.

Return type:

Quantile(X, q, axis=0)[source]¶

Compute quantiles of the data.

Parameters:

X (array-like) – Input data.
q (float or array-like) – Quantile(s) in [0, 1].
axis (int) – Axis to compute quantile along.

Returns:

Quantile values.

Return type:

scalar or array

Range(X, axis=0)[source]¶

Compute range (max - min) along an axis.

Parameters:

X (array-like) – Input data.
axis (int) – Axis along which to compute range.

Returns:

Range value(s).

Return type:

numpy.ndarray or scalar

RelativeFrequency(data, bins=10, returnFreqOnly=True, returnMean=False)[source]¶

Compute relative frequency using scipy.stats.relfreq.

Parameters:

data (array-like) – Input data.
bins (int) – Number of bins.
returnFreqOnly (bool) – If True return only frequency array.
returnMean (bool) – If True return the mean of results.

Returns:

Relative frequency outputs depending on flags.

Return type:

array or tuple

RowsMean(data)[source]¶

Compute the mean per row for a 2D array (asserts 2D input).

Parameters:: data (numpy.ndarray) – 2D input array.
Returns:: Mean per row.
Return type:: numpy.ndarray

SciPyFisherKurtosis(data, returnMean=False)[source]¶

Compute Fisher (excess) kurtosis via scipy.stats.kurtosis.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of kurtosis results.

Returns:

Kurtosis values.

Return type:

float or numpy.ndarray

SciPyFisherKurtosisDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper around scipy kurtosis (Fisher definition).

Parameters:

data (array-like) – Input samples.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Kurtosis values.

Return type:

numpy.ndarray or scalar

SciPyPearsonKurtosisDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper around scipy kurtosis (Pearson definition).

Parameters:

data (array-like) – Input samples.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Kurtosis values.

Return type:

numpy.ndarray or scalar

SciPyPearsonKurtosis(data, returnMean=False)[source]¶

Compute Pearson kurtosis via scipy.stats.kurtosis.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of kurtosis results.

Returns:

Kurtosis values.

Return type:

float or numpy.ndarray

HistPearsonKurtosis(pdf, range)[source]¶

Compute kurtosis from a histogram PDF and bin locations.

Parameters:

pdf (array-like) – Probability mass per bin.
range (array-like) – Bin centers.

Returns:

Pearson kurtosis estimate from histogram.

Return type:

SciPySkewnessDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic wrapper around scipy.stats.skew.

Parameters:

data (array-like) – Input samples.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Skewness values.

Return type:

numpy.ndarray or scalar

SciPySkewness(data, returnMean=False)[source]¶

Compute skewness using scipy.stats.skew.

Parameters:

data (array-like) – Input data.
returnMean (bool) – If True return mean of skewness results.

Returns:

Skewness values.

Return type:

float or numpy.ndarray

HistSkewness(pdf, range)[source]¶

Estimate skewness from a histogram PDF.

Parameters:

pdf (array-like) – Probability mass per bin.
range (array-like) – Bin center locations.

Returns:

Skewness of the histogram distribution.

Return type:

ShannonEntropy(data)[source]¶

Compute Shannon entropy using skimage.measure.shannon_entropy averaged over inputs.

Parameters:: data (array-like) – Input image or distribution to compute entropy for.
Returns:: Mean Shannon entropy.
Return type:: float

SkewnessDynamic(data, ddof=0, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Compute skewness using moment-based formula with flexible axes.

Parameters:

data (array-like) – Input samples.
ddof (int) – Delta degrees of freedom for variance.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Skewness values.

Return type:

numpy.ndarray or scalar

StandardDeviation(X, axis=0, ddof=0)[source]¶

Compute standard deviation.

Parameters:

X (array-like) – Input data.
axis (int) – Axis to compute along.
ddof (int) – Degrees of freedom correction.

Returns:

Standard deviation.

Return type:

numpy.ndarray or scalar

StandardDeviationDynamic(data, ddof=0, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶: Dynamic wrapper computing standard deviation via VarianceDynamic.

Sum(data)[source]¶

Compute the total sum of elements.

Parameters:: data (array-like) – Input data.
Returns:: Sum of all elements.
Return type:: scalar

SumDynamic(data, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶: Dynamic wrapper for np.sum.

TValueUsingTwoGroups(X, y)[source]¶

Compute an approximate independent two-sample t-value between two groups.

\[t \approx \frac{|\bar{x} - \bar{y}|}{\sqrt{\frac{s_x^2}{n_x} + \frac{s_y^2}{n_y}}}\]

Parameters:

X (array-like) – First sample array.
y (array-like) – Second sample array.

Returns:

t-statistic.

Return type:

Variance(X, axis=0, ddof=0)[source]¶

Compute variance along an axis.

Parameters:

X (array-like) – Input data.
axis (int) – Axis along which to compute.
ddof (int) – Degrees of freedom correction.

Returns:

Variance value(s).

Return type:

numpy.ndarray or scalar

HistVariance(pdf, range)[source]¶

Compute variance for a histogram-defined distribution.

Parameters:

pdf (array-like) – Probability mass per bin.
range (array-like) – Bin center locations.

Returns:

Variance of the histogram distribution.

Return type:

VarianceDynamic(data, ddof=0, returnMean=False, perLastAxis=False, axis=None, keepdims=False)[source]¶

Dynamic variance computation supporting flexible axes.

Parameters:

data (array-like) – Input samples.
ddof (int) – Delta degrees of freedom for variance.
returnMean (bool) – If True return mean of the result.
perLastAxis (bool) – If True operate along the last axis.
axis (int) – Axis to operate over.
keepdims (bool) – Whether to keep reduced dimensions.

Returns:

Variance values.

Return type:

numpy.ndarray or scalar

ZValueUsingTwoGroups(X, y, value=0.0)[source]¶

Compute Z-statistic for difference of two group means (assuming known population std).

\[z = \frac{|\bar{x} - \bar{y}| - v}{\sqrt{\frac{\sigma_x^2}{n_x} + \frac{\sigma_y^2}{n_y}}}\]

Parameters:

X (array-like) – First sample.
y (array-like) – Second sample.
value (float) – Hypothesized difference (default 0.0).

Returns:

z-statistic.

Return type:

ConfidenceInterval(data, confidence=0.95)[source]¶

Compute a Student-t based confidence interval for the mean.

Parameters:

data (array-like) – Sample observations.
confidence (float) – Confidence level (0-1).

Returns:

(lower bound, upper bound)

float: Lower bound of the confidence interval.
float: Upper bound of the confidence interval.

Return type:

class HMB.StatisticalAnalysisHelper.StatisticalAnalysisFramework(data, baseline=None, threshold=None, equivalenceMargin=None, alpha=0.05)[source]¶

Bases: object

A modular validator for repeated-trial system evaluation. Automatically selects, runs, and explains applicability of statistical methods. The framework is designed to be extensible, allowing for new methods and tests to be added as needed.

Mermaid diagram of the validation pipeline (text representation):

flowchart TD
  A["Start: Input Data<br>(baseline, threshold, etc.)"] --> B{Infer Data Structure}
  B -->|1D Array| C1["Set isMultiConfig = False<br>sampleSize = len(data)"]
  B -->|" 2D Array<br>(subjects × configs) "| C2["Set isMultiConfig = True<br>nConfigs = cols<br>sampleSize = rows"]
  C1 --> D{Infer Data Type}
  C2 --> D
  D -->|Binary| E1["Set dataType = Binary"]
  D -->|Continuous| E2["Set dataType = Continuous"]
  E1 --> F["Skip Normality/Symmetry Checks"]
  E2 --> G{Check Assumptions}
  G --> H1["Shapiro-Wilk (n ≤ 5000)?"]
  H1 -->|Yes| I1["Run Shapiro-Wilk<br>Set isNormal"]
  H1 -->|No| I2["Skip Shapiro-Wilk"]
  G --> H2["D'Agostino (n > 20)?"]
  H2 -->|Yes| I3["Run D'Agostino<br>Update isNormal if n>5000"]
  H2 -->|No| I4["Skip D'Agostino"]
  G --> H3["Lilliefors (n > 5000)?"]
  H3 -->|Yes| I5["Run Lilliefors<br>Set isNormal"]
  H3 -->|No| I6["Skip Lilliefors"]
  G --> H4["Baseline Provided?"]
  H4 -->|Yes| I7["Compute Skewness<br>Set isSymmetric"]
  H4 -->|No| I8["Assume Symmetric"]
  I1 --> J
  I2 --> J
  I3 --> J
  I4 --> J
  I5 --> J
  I6 --> J
  I7 --> J
  I8 --> J
  F --> J["Assess Method Applicability"]
  J --> K1{Has Threshold?}
  K1 -->|Yes| L1["Evaluate One-Sample Tests:<br>t-test, Wilcoxon, Sign, Binomial"]
  K1 -->|No| L2["Skip One-Sample Tests"]
  J --> K2{Has Baseline?}
  K2 -->|Yes| L3["Evaluate Paired Tests:<br>Paired t, Wilcoxon, Permutation,<br>McNemar, TOST"]
  K2 -->|No| L4["Skip Paired Tests"]
  J --> K3{isMultiConfig?}
  K3 -->|Yes| L5["Evaluate Multi-Config Tests:<br>- ICC (≥2)<br>- Friedman / RM-ANOVA (≥3)<br>- Cochran's Q (Binary, ≥3)"]
  K3 -->|No| L6["Skip Multi-Config Tests"]
  J --> K4{Has predictions & labels?}
  K4 -->|Yes| L7["Evaluate Calibration Tests:<br>CalibrationECE, HosmerLemeshow"]
  K4 -->|No| L8["Skip Calibration Tests"]
  J --> K5{Data Type?}
  K5 -->|Binary| L9["Evaluate Binomial Methods:<br>Clopper-Pearson, Beta-Binomial"]
  K5 -->|Continuous| L10["Evaluate CV, MAD, Effect Sizes"]
  L1 --> M
  L2 --> M
  L3 --> M
  L4 --> M
  L5 --> M
  L6 --> M
  L7 --> M
  L8 --> M
  L9 --> M
  L10 --> M
  M["Generate Method Assessments:<br>Applied/Not Applied + Reasons"] --> N["Execute Applicable Tests"]
  %% One-Sample Branch
  N --> O1{One-Sample Tests?}
  O1 -->|Yes| P1["Run: t-test / Wilcoxon / Sign / Binomial"]
  O1 -->|No| P2["Skip One-Sample Tests"]
  %% Paired Branch
  N --> O2{Paired Tests?}
  O2 -->|Yes| P3["Run: Paired t / Wilcoxon / Permutation /<br>McNemar / TOST"]
  O2 -->|No| P4["Skip Paired Tests"]
  %% Multi-Config Branch
  N --> O3{Multi-Config Tests?}
  O3 -->|Yes| P5["Run: ICC / Friedman / RM-ANOVA /<br>Cochran's Q"]
  O3 -->|No| P6["Skip Multi-Config Tests"]
  %% Calibration Branch
  N --> O4{Calibration Tests?}
  O4 -->|Yes| P7["Run: CalibrationECE / HosmerLemeshow"]
  O4 -->|No| P8["Skip Calibration Tests"]
  %% Descriptive Stats
  N --> O5{Descriptive Stats?}
  O5 -->|Yes| P9["Compute: CV, MAD"]
  O5 -->|No| P10["Skip Descriptive Stats"]
  %% Effect Size
  N --> O6{Effect Size?}
  O6 -->|Yes| P11["Compute: Cohen's d / Hedges' g / A12"]
  O6 -->|No| P12["Skip Effect Size"]
  %% Bootstrap
  N --> O7{Bootstrap?}
  O7 -->|Yes| P13["Compute BCa CIs:<br>Mean/Median/Proportion/5th %ile"]
  O7 -->|No| P14["Skip Bootstrap (should not occur)"]
  %% Tolerance Interval
  N --> O8{Tolerance Interval?}
  O8 -->|Yes| P15["Compute Bootstrap Approximation"]
  O8 -->|No| P16["Skip Tolerance Interval"]
  %% Extra Normality Tests
  N --> O9{Normality Tests?}
  O9 -->|Yes| P17["Run: Anderson-Darling / Jarque-Bera"]
  O9 -->|No| P18["Skip Extra Normality Tests"]
  %% Converge all paths
  P1 --> Q
  P2 --> Q
  P3 --> Q
  P4 --> Q
  P5 --> Q
  P6 --> Q
  P7 --> Q
  P8 --> Q
  P9 --> Q
  P10 --> Q
  P11 --> Q
  P12 --> Q
  P13 --> Q
  P14 --> Q
  P15 --> Q
  P16 --> Q
  P17 --> Q
  P18 --> Q
  Q["Compile Results + Execution Log"] --> R["Return Results Dictionary"]
  R --> S["End"]

Examples

from HMB.StatisticalAnalysisHelper import StatisticalAnalysisFramework

# Example usage:
data = [0.8, 0.85, 0.9, 0.95, 0.92]  # Sample performance data.
baseline = [0.75, 0.8, 0.78, 0.82, 0.79]  # Sample baseline data (e.g., human performance).
threshold = 0.8  # Performance threshold (e.g., above chance).
equivalenceMargin = 0.05  # Margin for equivalence testing.

validator = StatisticalAnalysisFramework(
  data,
  baseline=baseline,
  threshold=threshold,
  equivalenceMargin=equivalenceMargin
)
results = validator.RunValidation()
print(results)

Initialize the StatisticalAnalysisFramework with input data and parameters.

Parameters:

data (array-like) – The primary dataset to analyze (e.g., performance metrics).
baseline (array-like, optional) – A reference dataset for comparison (e.g., human performance).
threshold (float, optional) – A performance threshold for certain tests (e.g., above chance).
equivalenceMargin (float, optional) – Margin for equivalence testing.
alpha (float, optional) – Significance level for hypothesis testing (default 0.05).

This constructor sets up the internal state of the validator, including storing the input data, baseline, threshold, equivalence margin, and significance level. It also prepares structures for results and logging decisions throughout the validation process.

__init__(data, baseline=None, threshold=None, equivalenceMargin=None, alpha=0.05)[source]¶

Initialize the StatisticalAnalysisFramework with input data and parameters.

Parameters:

data (array-like) – The primary dataset to analyze (e.g., performance metrics).
baseline (array-like, optional) – A reference dataset for comparison (e.g., human performance).
threshold (float, optional) – A performance threshold for certain tests (e.g., above chance).
equivalenceMargin (float, optional) – Margin for equivalence testing.
alpha (float, optional) – Significance level for hypothesis testing (default 0.05).

RunValidation()[source]¶

Execute the full validation process, including data type inference, assumption checks, method applicability evaluation, method assessment, test execution, and result compilation.

Returns:: A comprehensive results dictionary containing test outcomes, execution log, and method assessments.
Return type:: dict

InferDataType()[source]¶: Infer the data type (Binary vs Continuous) based on unique values in the input data.

CheckAssumptions()[source]¶

Check statistical assumptions such as normality and symmetry based on data type and sample size.

For Continuous data:

Normality is assessed using Shapiro-Wilk for n <= 5000, D’Agostino’s K^2 for n > 20, and Lilliefors for n > 5000.
Symmetry is assessed via skewness of differences from baseline when a baseline is provided (paired design).
Logs are generated for each test applied or skipped with reasons.

For Binary data:

Normality is not applicable and set to False.
Symmetry is treated as True by default.
Logs are generated indicating that normality and symmetry checks were skipped for binary data.

EvaluateMethodApplicability()[source]¶

Evaluate the applicability of various statistical methods based on the inferred data type, sample size, and assumption checks.

For Continuous data, applicability of normality tests, one-sample tests (t-test, Wilcoxon, Sign), paired
tests (Paired t-test, Wilcoxon signed-rank, Permutation), and effect size calculations (Cohen’s d, Hedges’ g) are determined based on normality, symmetry, presence of baseline, and sample size.
For Binary data, applicability of methods like Exact Binomial Test, Clopper-Pearson CI, Bayesian Beta-Binomial,
McNemar’s test, and Cochran’s Q is determined based on the presence of a threshold, baseline, and whether the data is multi-config.
Logs are generated for each method indicating whether it is applicable or not, along with the reasons based
on the data characteristics and assumptions.
The results of this evaluation are stored in an applicableMethods dictionary mapping method names to
boolean flags indicating applicability.

GetApplyReason(methodName)[source]¶

Get a human-readable reason for why a method is applied based on the data characteristics and assumptions.

Parameters:: methodName (str) – The name of the method for which to generate the apply reason.
Returns:: A human-readable explanation for why the method is applied.
Return type:: str

AssessMethods()[source]¶

Assess each method’s applicability and produce a mapping of method names to whether they were applied or skipped, along with human-readable reasons for each decision.

Key points:

Iterates over the applicableMethods dictionary to determine which methods are applied or skipped.
For applied methods, uses GetApplyReason to generate a reason and records it in methodAssessments.
For skipped methods, uses GetSkipReason to generate a reason and records it in methodAssessments.
Logs the decision for each method in the execution log for transparency and traceability.
The resulting methodAssessments dictionary maps each method name to a dict with keys "Applied" and "Reason".

This structured assessment provides clear communication of which methods were used and why, and which were not used.

GetSkipReason(methodName)[source]¶

Get a human-readable reason for why a method is skipped based on the data characteristics and assumptions.

Parameters:: methodName (str) – The name of the method for which to generate the skip reason.
Returns:: A human-readable explanation for why the method is skipped.
Return type:: str

SelectAndRunTests()[source]¶

Select and execute the applicable statistical tests based on the applicableMethods mapping, and populate the results dictionary with test outcomes.

Iterates over the applicableMethods to identify which tests are marked as applicable.
For each applicable test, the corresponding statistical test is executed using SciPy or custom implementations as needed.
The results of each test (e.g., test statistics, p-values, confidence intervals) are stored in the results dictionary under keys corresponding to each test.
For one-sample tests, if a threshold is provided, the relevant tests are executed and their results are stored under a “OneSampleTests” key in the results.
For multi-configuration tests, the appropriate methods are executed and results are stored under keys corresponding to each method.
For effect size calculations and variability metrics, the computed values are stored under descriptive keys in the results.
The method assessments are included in the results for transparency, allowing users to see which methods were applied and the reasons for their application or exclusion.

ComputeBCaBootstrap(nBoot=5000, ciLevel=0.95)[source]¶

Compute bias-corrected and accelerated (BCa) bootstrap confidence intervals for key statistics based on the provided data.

This method implements the BCa bootstrap procedure to compute confidence intervals for statistics such as mean, median, and proportions.
It generates bootstrap samples by resampling the original data with replacement and computes the statistic of interest on each bootstrap sample to create a distribution of bootstrap estimates.
The bias-correction (z0) and acceleration (acc) parameters are computed using the bootstrap distribution and jackknife replicates, respectively, to adjust the percentile positions for the confidence interval.
The method returns a dictionary containing the point estimate and the lower and upper bounds of the BCa confidence interval for each relevant statistic, formatted with CamelCase keys for clarity.

Parameters:

nBoot (int, optional) – Number of bootstrap resamples to perform. Default is 5000.
ciLevel (float, optional) – Confidence level for the intervals (e.g., 0.95 for 95% CI). Default is 0.95.

Returns:

A dictionary containing the BCa confidence intervals and point estimates for relevant statistics based on the data type (proportion for binary data, mean/median/fifth percentile for continuous data).

Return type:

dict

HMB.StatisticalAnalysisHelper.CohensDPaired(a, b, ddof=1)[source]¶

Compute Cohen’s d for paired samples (mean of differences / sd of differences). Handles edge cases by returning NaN when standard deviation of differences is zero or when either input array is empty.

Parameters:

a (array-like) – First set of paired observations.
b (array-like) – Second set of paired observations.
ddof (int, optional) – Degrees of freedom for standard deviation (default is 1 for sample standard deviation).

Returns:

Cohen’s d value for the paired samples. Returns NaN if standard deviation of differences is zero or if either input array is empty.

Return type:

HMB.StatisticalAnalysisHelper.BenjaminiHochberg(pvals)[source]¶

Simple BH FDR correction for a flat list/array of p-values. Returns an array of adjusted p-values in the same order as input.

Parameters:: pvals (array-like) – List or array of p-values to adjust for multiple comparisons.
Returns:: Array of adjusted p-values corresponding to the input p-values, controlling the false
Return type:: np.ndarray

HMB.StatisticalAnalysisHelper.StatisticalAnalysis(results, hypothesizedMean=0, secondMetricList=None, confidenceLevel=0.95, nBootstraps=1000)[source]¶

Perform comprehensive statistical analysis on a list of results.

Parameters:

results (list) – List of performance metrics (e.g., accuracy, scores).
hypothesizedMean (float, optional) – Mean value for one-sample t-test. Default is 0.
secondMetricList (list, optional) – Second list of metrics for correlation/regression analysis. Default is None.
confidenceLevel (float, optional) – Confidence level for confidence intervals. Default is 0.95.
nBootstraps (int, optional) – Number of bootstrap resamples for confidence intervals. Default is 1000.

Returns:

Dictionary containing all statistical analysis results.

Return type:

dict

Examples

import HMB.StatisticalAnalysisHelper as sah

results = [0.8, 0.82, 0.78, 0.81, 0.79]
analysisReport = sah.StatisticalAnalysis(results, hypothesizedMean=0.75)
print("Statistical Analysis Report:")
print(analysisReport)

HMB.StatisticalAnalysisHelper.ExtractDataFromSummaryFile(file)[source]¶

Extract and organize data from a summary CSV file containing names, metrics, and trial data. The file is expected to be structured as follows:

First line: Comma-separated names (headers for individuals or categories).
Second line: Comma-separated metrics (headers for performance or evaluation criteria).
Subsequent lines: Numerical values corresponding to metrics for each trial.

Example of the file structure (if you have multiple systems):

System A, , , , , , System B, , , , , Precision, Recall, F1, Accuracy, Specificity, Average, Precision, Recall, F1, Accuracy, Specificity, Average 0.5556, 0.5556, 0.5556, 0.6667, 0.7333, 0.6133, 0.5556, 0.5556, 0.5556, 0.6667, 0.7333, 0.6133 0.7692, 0.5556, 0.6452, 0.7708, 0.9000, 0.7282, 0.7692, 0.5556, 0.6452, 0.7708, 0.9000, 0.7282 0.8333, 0.2778, 0.4167, 0.7083, 0.9667, 0.6406, 0.8333, 0.2778, 0.4167, 0.7083, 0.9667, 0.6406 0.5882, 0.5556, 0.5714, 0.6875, 0.7667, 0.6339, 0.5882, 0.5556, 0.5714, 0.6875, 0.7667, 0.6339 0.8000, 0.6667, 0.7273, 0.8125, 0.9000, 0.7813, 0.8000, 0.6667, 0.7273, 0.8125, 0.9000, 0.7813

Example of the file structure (if you have a single system):

Precision, Recall, F1, Accuracy, Specificity, Average Metric, Metric, Metric, Metric, Metric, Metric 0.5556, 0.5556, 0.5556, 0.6667, 0.7333, 0.6133, 0.7692, 0.5556, 0.6452, 0.7708, 0.9000, 0.7282, 0.8333, 0.2778, 0.4167, 0.7083, 0.9667, 0.6406, 0.5882, 0.5556, 0.5714, 0.6875, 0.7667, 0.6339, 0.8000, 0.6667, 0.7273, 0.8125, 0.9000, 0.7813,

Parameters:

file (str) – Path to the summary CSV file.

Returns:

A tuple containing:

history (list): A list of dictionaries, each representing a name with its metrics and trial data.
names (list): A list of cleaned names extracted from the file.
metrics (list): A list of cleaned metrics extracted from the file.

Return type: