A Functional Inspired by Min-Max Normalization

Math
Functions
Functionals
Author

Galen Seilis

Published

March 19, 2023

For a function \(F: \mathbb{R}^n \mapsto \mathbb{R}\) we can consider the functional

\[\frac{F(x) - F(a)}{F(b) - F(a)} \]

where \[a = \arg\min_x F(x)\] and \[b = \arg\max_x F(x)\]. You may recognize this to be the form of min-max normalization. If we restrict \(F\) to be a cumulative distribution function then this is also the form of a truncated probability distribution (Casella and Berger 2002). Both of these notions are useful for certain kinds of modelling problems. For recreation rather than utility, I generalize this functional below.

We can consider a more general functional

\[g \left( f\left( F(x), F(a) \right), f\left( F(b), F(a) \right) \right)\]

where \(f\) and \(g\) are suitably-defined binary operators.

Taking common operations including addition, subtraction, multiplication, division, exponentiation, and logarithms, I’ve tabulated various choices below.

f g Functional
add add \(2 F{\left(a \right)} + F{\left(b \right)} + F{\left(x \right)}\)
add subtract \(- F{\left(b \right)} + F{\left(x \right)}\)
add mult \(\left(F{\left(a \right)} + F{\left(b \right)}\right) \left(F{\left(a \right)} + F{\left(x \right)}\right)\)
add divide \(\frac{F{\left(a \right)} + F{\left(x \right)}}{F{\left(a \right)} + F{\left(b \right)}}\)
add exp \(\left(F{\left(a \right)} + F{\left(x \right)}\right)^{F{\left(a \right)} + F{\left(b \right)}}\)
add log \(\frac{\log{\left(F{\left(a \right)} + F{\left(b \right)} \right)}}{\log{\left(F{\left(a \right)} + F{\left(x \right)} \right)}}\)
subtract add \(- 2 F{\left(a \right)} + F{\left(b \right)} + F{\left(x \right)}\)
subtract subtract \(- F{\left(b \right)} + F{\left(x \right)}\)
subtract mult \(\left(F{\left(a \right)} - F{\left(b \right)}\right) \left(F{\left(a \right)} - F{\left(x \right)}\right)\)
subtract divide \(\frac{F{\left(a \right)} - F{\left(x \right)}}{F{\left(a \right)} - F{\left(b \right)}}\)
subtract exp \(\left(- F{\left(a \right)} + F{\left(x \right)}\right)^{- F{\left(a \right)} + F{\left(b \right)}}\)
subtract log \(\frac{\log{\left(- F{\left(a \right)} + F{\left(b \right)} \right)}}{\log{\left(- F{\left(a \right)} + F{\left(x \right)} \right)}}\)
mult add \(\left(F{\left(b \right)} + F{\left(x \right)}\right) F{\left(a \right)}\)
mult subtract \(\left(- F{\left(b \right)} + F{\left(x \right)}\right) F{\left(a \right)}\)
mult mult \(F^{2}{\left(a \right)} F{\left(b \right)} F{\left(x \right)}\)
mult divide \(\frac{F{\left(x \right)}}{F{\left(b \right)}}\)
mult exp \(\left(F{\left(a \right)} F{\left(x \right)}\right)^{F{\left(a \right)} F{\left(b \right)}}\)
mult log \(\frac{\log{\left(F{\left(a \right)} F{\left(b \right)} \right)}}{\log{\left(F{\left(a \right)} F{\left(x \right)} \right)}}\)
divide add \(\frac{F{\left(b \right)} + F{\left(x \right)}}{F{\left(a \right)}}\)
divide subtract \(\frac{- F{\left(b \right)} + F{\left(x \right)}}{F{\left(a \right)}}\)
divide mult \(\frac{F{\left(b \right)} F{\left(x \right)}}{F^{2}{\left(a \right)}}\)
divide divide \(\frac{F{\left(x \right)}}{F{\left(b \right)}}\)
divide exp \(\left(\frac{F{\left(x \right)}}{F{\left(a \right)}}\right)^{\frac{F{\left(b \right)}}{F{\left(a \right)}}}\)
divide log \(\frac{\log{\left(\frac{F{\left(b \right)}}{F{\left(a \right)}} \right)}}{\log{\left(\frac{F{\left(x \right)}}{F{\left(a \right)}} \right)}}\)
exp add \(F^{F{\left(a \right)}}{\left(b \right)} + F^{F{\left(a \right)}}{\left(x \right)}\)
exp subtract \(- F^{F{\left(a \right)}}{\left(b \right)} + F^{F{\left(a \right)}}{\left(x \right)}\)
exp mult \(F^{F{\left(a \right)}}{\left(b \right)} F^{F{\left(a \right)}}{\left(x \right)}\)
exp divide \(F^{- F{\left(a \right)}}{\left(b \right)} F^{F{\left(a \right)}}{\left(x \right)}\)
exp exp \(\left(F^{F{\left(a \right)}}{\left(x \right)}\right)^{F^{F{\left(a \right)}}{\left(b \right)}}\)
exp log \(\frac{\log{\left(F^{F{\left(a \right)}}{\left(b \right)} \right)}}{\log{\left(F^{F{\left(a \right)}}{\left(x \right)} \right)}}\)
log add \(\frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(x \right)} \right)}} + \frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(b \right)} \right)}}\)
log subtract \(\frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(x \right)} \right)}} - \frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(b \right)} \right)}}\)
log mult \(\frac{\log{\left(F{\left(a \right)} \right)}^{2}}{\log{\left(F{\left(b \right)} \right)} \log{\left(F{\left(x \right)} \right)}}\)
log divide \(\frac{\log{\left(F{\left(b \right)} \right)}}{\log{\left(F{\left(x \right)} \right)}}\)
log exp \(\left(\frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(x \right)} \right)}}\right)^{\frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(b \right)} \right)}}}\)
log log \(\frac{\log{\left(\frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(b \right)} \right)}} \right)}}{\log{\left(\frac{\log{\left(F{\left(a \right)} \right)}}{\log{\left(F{\left(x \right)} \right)}} \right)}}\)
Warning

Note that notation such as \(F^2(a)\) and \(F^{F(a)}(b)\) denote exponentiation, not powers of function composition.

If you are interested in tinkering with your own choice of functions, here is the source code to generate the above table.

"""Generate the functional table shown in the blog post."""

from __future__ import annotations

from itertools import product
from typing import Callable

import pandas as pd
import sympy as sp


def generate_functional_table() -> pd.DataFrame:
    """Return a DataFrame with (f, g, Functional) as markdown-formatted latex.

    The operations are over F(x), F(a), F(b) as in the original post.
    """
    x = sp.Symbol("x")
    b = sp.Symbol("b")
    a = sp.Symbol("a")
    F = sp.Function("F")
    F_x = F(x)
    F_b = F(b)
    F_a = F(a)

    ops: dict[str, Callable[[sp.Expr, sp.Expr], sp.Expr]] = {
        "add": lambda x_, y_: x_ + y_,
        "subtract": lambda x_, y_: x_ - y_,
        "mult": lambda x_, y_: x_ * y_,
        "divide": lambda x_, y_: x_ / y_,
        "exp": lambda x_, y_: x_**y_,
        "log": lambda x_, y_: sp.log(y_, x_),
    }

    results = []
    for f, g in product(ops, ops):
        result = ops[g](ops[f](F_x, F_a), ops[f](F_b, F_a))
        result = result.simplify()
        results.append([f, g, f"${sp.latex(result)}$"])

    df = pd.DataFrame(results, columns=["f", "g", "Functional"])
    return df

At a glance it may appear that our original functional is absent in the table, but noting that

\[\frac{F{\left(a \right)} - F{\left(x \right)}}{F{\left(a \right)} - F{\left(b \right)}} = \frac{-1}{-1} \frac{F(x) - F(a)}{F(b) - F(a)}\]

reveals that SymPy produced the same result but with a distinct expression.

It is easy to confirm that not all of the functionals I have tabulated above will provide a normalization onto the interval \([0,1]\). Because this and other properties may not be preserved, one might regard this as an “unnatural generalization”.

References

Casella, George, and Roger L. Berger. 2002. Statistical Inference. 2nd ed. Duxbury.