Synopsis

#include <boost/math/statistics/chatterjee_correlation.hpp>

namespace boost::math::statistics {

    C++17:
    template <typename ExecutionPolicy, typename Container>
    auto chatterjee_correlation(ExecutionPolicy&& exec, const Container& u, const Container& v);

    C++11:
    template <typename Container>
    auto chatterjee_correlation(const Container& u, const Container& v);
}

Description

The classical correlation coefficients like the Pearson's correlation are useful primarily for distinguishing when one dataset depends linearly on another. However, Pearson's correlation coefficient has a known weakness in that when the dependent variable has an obvious functional relationship with the independent variable, the value of the correlation coefficient can take on any value. As Chatterjee says:

> Ideally, one would like a coefficient that approaches its maximum value if and only if one variable looks more and more like a noiseless function of the other, just as Pearson correlation is close to its maximum value if and only if one variable is close to being a noiseless linear function of the other.

This is the problem Chatterjee's coefficient solves. Let X and Y be random variables, where Y is not constant, and let (X_i, Y_i) be samples from this distribution. Rearrange these samples so that X_(0) < X_{(1)} < ... X_{(n-1)} and create (R(X_{(i)}), R(Y_{(i)})). The Chatterjee correlation is then given by

In the limit of an infinite amount of i.i.d data, the statistic lies in [0, 1]. However, if the data is not infinite, the statistic may be negative. If X and Y are independent, the value is zero, and if Y is a measurable function of X, then the statistic is unity. The complexity is O(n log n).

An example is given below:

std::vector<double> X{1,2,3,4,5};
std::vector<double> Y{1,2,3,4,5};
using boost::math::statistics::chatterjee_correlation;
double coeff = chatterjee_correlation(X, Y);

The implementation follows Chatterjee's paper.

Nota bene: If the input is an integer type the output will be a double precision type.

Invariants

The function expects at least two samples, a non-constant vector Y, and the same number of X's as Y's. If Y is constant, the result is a quiet NaN. The data set must be sorted by X values. If there are ties in the values of X, then the statistic is random due to the random breaking of ties. Of course, random numbers are not used internally, but the result is not guaranteed to be identical on different systems.

References