Bayes
Gaussian, multinomial and Bernoulli naive Bayes classifiers, fitted in closed form, plus per-class sizes, priors, means and variances.
This module provides the three standard naive Bayes classifiers: Gaussian (for continuous features), multinomial (for counts, such as word counts in documents) and Bernoulli (for yes/no features). “Naive” refers to the assumption that features are independent within a class. The assumption is rarely true, yet the classifiers work well in practice, and it is what makes them cheap: the likelihood factorises over columns, so no covariance matrix has to be inverted.
Every fit here is closed form. There is no loop, no convergence and no learning rate: the per-class statistics are contractions of the data against the class membership matrix, and the scores for every query and every class are one more contraction. Each classifier takes the training data and the queries together and returns scores or predictions directly; there is no separate model object.
Conventions. Class membership g is a one-hot matrix n×K (row i has a 1 in the column of its class). Training data x is n×p and queries z are m×p. Scores are m×K log-probabilities up to a constant per row. Predictions are 0/1 masks m×K; if two classes tie on the highest score, both are marked. For the multinomial and Bernoulli models, a is the Laplace (additive) smoothing count, typically 1, which stops a word never seen in a class from making that class impossible.
Honest limitation. Class variances are computed as \(E[x^2] - E[x]^2\) rather than by centring first, and floored at \(10^{-9}\). For features whose values are large compared with their spread (for example timestamps or values around \(10^6\) varying by units), this loses digits; centre or standardise such features before fitting.
open Prelude.Bayes
What every one of them needs
The per-class statistics shared by the classifiers. With \(n_k = \sum_i g_{ik}\) the size of class \(k\):
\[\pi_k = \frac{n_k}{n}, \qquad \mu_{kj} = \frac{1}{n_k}\sum_i g_{ik}\, x_{ij}, \qquad \sigma^2_{kj} = \frac{1}{n_k}\sum_i g_{ik}\, x_{ij}^2 - \mu_{kj}^2\]classSizes
classSizes(g) · g one-hot membership n×K → class sizes K
Let \(G \in \{0, 1\}^{n \times K}\) (the argument g) be the one-hot class membership, with \(G_{ik} = 1\) exactly when row \(i\) is in class \(k\). Then, for \(k = 1, \dots, K\),
The number of rows in each class.
Intuition Adding up the one-hot membership column by column counts the rows in each class. It is the denominator behind every per-class average on this page.
Use it when you check class balance before training, or need the class counts for smoothing or for weighting a metric. A badly imbalanced count warns that accuracy will be dominated by the large class. A class with no rows still gets a column of scores downstream, so remove empty columns from g before fitting.
class-imbalanceclassification
def classSizes(g) ≔ einsum("ik->k", g)
classPriors
classPriors(g) · g one-hot membership n×K → prior probabilities K
Let \(G \in \{0, 1\}^{n \times K}\) (the argument g) be the one-hot class membership, with \(G_{ik} = 1\) exactly when row \(i\) is in class \(k\). Then
The fraction of rows in each class, \(n_k/n\).
Intuition How common each class is before any feature is looked at. In a naive Bayes score \(\ln \pi_k\) is a head start, so a rare class needs much stronger evidence to win.
Use it when you need base rates, or the accuracy of the trivial “always predict the commonest class” baseline (its largest entry). The scorers here always use these training frequencies. If the class mix in use differs from the training data (for example after rebalancing), add \(\ln \pi'_k - \ln \pi_k\) to column \(k\) of the scores to impose the priors \(\pi'\) you believe.
class-imbalanceprobability-calibrationclassification
def classPriors(g) ≔ classSizes(g) / rows(g)
classMeans
classMeans(g, x) · g one-hot membership n×K · x data n×p → class means K×p
Let \(G \in \{0, 1\}^{n \times K}\) (the argument g) be the one-hot class membership, with \(G_{ik} = 1\) exactly when row \(i\) is in class \(k\), and let \(X \in \mathbb{R}^{n \times p}\) (the argument x) be the training data. Let \(n_k = \sum_i G_{ik}\). Then
The mean of every column within every class. An empty class gives a row of zeros rather than a division by zero.
Intuition The centre of each class, column by column: one representative point per class. Gaussian naive Bayes and the nearest-centroid classifier both judge a query by how far it lies from these rows.
Use it when you summarise groups (average spend per customer segment), build centroids for centroidPredict in Prelude.Neighbours, or look for the features that separate classes: columns where the rows differ a lot compared with the within-class spread from classVariances. An empty class gives a row of zeros, not an error.
comparing-groupsclassification
def classMeans(g, x) ≔ classSums(g, x) / safe(classSpread(g, x))
classVariances
classVariances(g, x) · g one-hot membership n×K · x data n×p → class variances K×p
Let \(G \in \{0, 1\}^{n \times K}\) (the argument g) be the one-hot class membership, with \(G_{ik} = 1\) exactly when row \(i\) is in class \(k\), and let \(X \in \mathbb{R}^{n \times p}\) (the argument x) be the training data. Let \(n_k = \sum_i G_{ik}\) and \(\mu_{kj} = \operatorname{classMeans}(G, X)_{kj}\). Then
The variance of every column within every class, dividing by the class size (not size minus one), floored at \(10^{-9}\) so a constant column does not cause a division by zero later.
Intuition How spread out each feature is inside each class. Gaussian naive Bayes divides squared deviations by it, so a feature that is tight within a class is strong evidence and a loose one is weak.
Use it when you fit Gaussian naive Bayes or compare within-class spread. It divides by \(n_k\), which understates spread in very small classes. A feature that is constant inside a class (common with few rows or discrete values) gets the floor \(10^{-9}\), and any query that differs on it is all but ruled out of that class. Centre or standardise large-valued features first, as the limitation above explains.
parameter-estimationnumerical-stabilityclassification
def classVariances(g, x) ≔ max(classSquares(g, x) − classMeans(g, x) · classMeans(g, x), 1e-9)
Building blocks
classSpread(g, x)— Let \(G \in \{0,1\}^{n \times K}\) be the one-hot membership and \(X \in \mathbb{R}^{n \times p}\). Then \(\operatorname{classSpread}(G, X)_{kj} = \sum_{i=1}^{n} G_{ik} = n_k\) for every \(j = 1, \dots, p\). Class sizes repeated across every column, shape K×p, so they divide K×p matrices correctly.classSums(g, x)— With \(G\) and \(X\) as above, \(\operatorname{classSums}(G, X)_{kj} = \sum_{i=1}^{n} G_{ik} X_{ij}\). The sum of every column within every class, shape K×p.classSquares(g, x)— With \(G\), \(X\) and \(n_k\) as above, \(\operatorname{classSquares}(G, X)_{kj} = \sum_{i} G_{ik} X_{ij}^2 / (n_k + [\,n_k = 0\,])\). The mean of every squared column within every class, shape K×p.
Gaussian naive Bayes
Each feature is modelled as normal within each class, independently of the others. The log joint density of query \(z_i\) and class \(k\) is
\[s_{ik} = \ln \pi_k - \frac12 \sum_j \ln(2\pi\sigma^2_{kj}) - \frac12 \sum_j \frac{(z_{ij} - \mu_{kj})^2}{\sigma^2_{kj}}\]computed for every query and every class at once from an m×K×p tensor of deviations.
gaussianScores
gaussianScores(g, x, z) · g one-hot membership n×K · x training data n×p · z queries m×p → log scores m×K
Let \(G \in \{0, 1\}^{n \times K}\) (the argument g) be the one-hot class membership, with \(G_{ik} = 1\) exactly when row \(i\) is in class \(k\), and let \(X \in \mathbb{R}^{n \times p}\) (the argument x) be the training data, let \(Z \in \mathbb{R}^{m \times p}\) (the argument z) be the queries, and let \(\pi_k\), \(\mu_{kj}\) and \(\sigma^2_{kj}\) be classPriors, classMeans and classVariances of \((G, X)\). Then
The log joint density \(s_{ik}\) of every query with every class, using the class priors, means and variances of the training data.
Intuition For each query and class: the log prior plus the log of a product of one-dimensional bell curves, one per feature. Each feature adds its evidence independently, and being many standard deviations from the class mean costs quadratically.
Use it when you need the raw log scores: to add other evidence, change the priors, or apply your own decision rule such as unequal misclassification costs. Only differences between classes in the same row carry meaning. Features should be continuous and roughly bell-shaped within a class, so log-transform skewed ones such as incomes. It builds an m×K×p tensor. For labels use gaussianPredict; for probabilities gaussianPosterior.
classificationcost-sensitive-decisions
def gaussianScores(g, x, z) ≔ liftedOverRows(z, ln(max(classPriors(g), 1e-300)) − gaussianLogDet(classVariances(g, x)) / 2) − gaussianQuad(z, classMeans(g, x), classVariances(g, x)) / 2
gaussianPredict
gaussianPredict(g, x, z) · g one-hot membership n×K · x training data n×p · z queries m×p → 0/1 mask m×K
Let \(G\), \(X\), \(Z\) be as in gaussianScores and \(s = \operatorname{gaussianScores}(G, X, Z) \in \mathbb{R}^{m \times K}\). Then
Hilbert returns a mask, not a class index.
The most probable class of each query, as a one-hot row. A tie marks every tied class.
Intuition Picks the class with the highest posterior probability. The evidence term is the same for every class in a row, so that is simply the class with the highest log score.
Use it when you want a fast baseline for continuous features and little data: it fits in closed form, has nothing to tune, and often works reasonably even though features are rarely independent. The output is a one-hot mask, and ties mark several classes. When features are strongly correlated, try the shared-covariance discriminant in Prelude.Multivariate; for counts or yes/no features use multinomialPredict or bernoulliPredict.
classificationbaseline-comparison
def gaussianPredict(g, x, z) ≔ highest(gaussianScores(g, x, z))
gaussianPosterior
gaussianPosterior(g, x, z) · g one-hot membership n×K · x training data n×p · z queries m×p → probabilities m×K
Let \(G\), \(X\), \(Z\) be as in gaussianScores and \(s = \operatorname{gaussianScores}(G, X, Z)\). Then
The posterior probability of each class for each query: the row-wise softmax of the scores. Each row sums to one.
Intuition The softmax turns log scores into probabilities by exponentiating and dividing by their sum, after subtracting the row maximum so nothing overflows. Naive Bayes counts correlated evidence several times, so these probabilities tend to be too extreme, crowding toward 0 and 1.
Use it when you need class probabilities for ranking, custom thresholds or expected-cost decisions. Treat them as scores unless you have checked their calibration on held-out data. If calibrated probabilities matter, recalibrate them or use a discriminative model such as logisticFit in Prelude.Regression.
probability-calibrationuncertainty-quantificationclassification
def gaussianPosterior(g, x, z) ≔ softmaxRows(gaussianScores(g, x, z))
Building blocks
spanRows(z, m)— Let \(Z \in \mathbb{R}^{m \times p}\) be the queries and \(M \in \mathbb{R}^{K \times p}\) (the argumentm) the class means. Then \(\operatorname{spanRows}(Z, M) \in \mathbb{R}^{m \times K \times p}\) with \(\operatorname{spanRows}(Z, M)_{ikj} = Z_{ij}\). Repeats each query row across every class.spanMeans(z, m)— With \(Z\) and \(M\) as above, \(\operatorname{spanMeans}(Z, M)_{ikj} = M_{kj}\). Repeats each class mean across every query: shape m×K×p.deviations3(z, m)— With \(Z\) and \(M\) as above, \(\operatorname{deviations3}(Z, M)_{ikj} = D_{ikj} = Z_{ij} - M_{kj}\). The deviation of every query from every class mean in every column, m×K×p.gaussianQuad(z, m, v)— Let \(Z\), \(M\) and \(D\) be as above and \(V \in \mathbb{R}^{K \times p}\) (the argumentv) the class variances, \(V_{kj} > 0\). Then \(\operatorname{gaussianQuad}(Z, M, V)_{ik} = \sum_{j=1}^{p} D_{ikj}^2 / V_{kj}\). The sum over columns of squared deviations divided by class variances, m×K.gaussianLogDet(v)— Let \(V \in \mathbb{R}^{K \times p}\) with \(V_{kj} > 0\). Then \(\operatorname{gaussianLogDet}(V)_k = \sum_{j=1}^{p} \ln(2\pi V_{kj})\). The per-class sum over columns of the log normaliser, shape K.liftedOverRows(z, u)— Let \(Z \in \mathbb{R}^{m \times p}\) and \(u \in \mathbb{R}^K\). Then \(\operatorname{liftedOverRows}(Z, u)_{ik} = u_k\) for \(i = 1, \dots, m\). Repeats a per-class vector across every query row, shape m×K.
Multinomial naive Bayes
The model for counts. Each document (row) is a vector of term counts, and class \(k\) has a probability \(\theta_{kj}\) for each term, estimated with additive smoothing:
\[\theta_{kj} = \frac{N_{kj} + a}{\sum_l N_{kl} + p\,a}, \qquad s_{ik} = \ln \pi_k + \sum_j z_{ij} \ln \theta_{kj}\]where \(N_{kj}\) is the total count of term \(j\) in class \(k\). The multinomial coefficient is the same for every class and is omitted. Here x and z are count matrices, not design matrices.
wordShares
wordShares(g, x, a) · g one-hot membership n×K · x training counts n×p · a smoothing count scalar → term probabilities K×p
Let \(G \in \{0, 1\}^{n \times K}\) (the argument g) be the one-hot class membership, with \(G_{ik} = 1\) exactly when row \(i\) is in class \(k\), let \(X \in \mathbb{R}^{n \times p}\) (the argument x) be the training counts, and let \(a \ge 0\) be the smoothing count. Let \(N_{kj} = \sum_i G_{ik} X_{ij}\) and \(T_k = \sum_{l=1}^{p} (N_{kl} + a)\). Then
The smoothed probability \(\theta_{kj}\) of each term in each class; each row sums to one.
Intuition Each class’s word distribution: the share of all word occurrences in that class that are word \(j\). Adding a pretends every word was seen a extra times in every class, so a word never seen in a class gets a small probability instead of zero.
Use it when you inspect what characterises a class (the largest entries of a row, or the words whose share differs most between classes) or need the fitted parameters of multinomial naive Bayes. a = 1 is Laplace smoothing; smaller values such as 0.1 trust the counts more when the corpus is large.
text-classificationkeyword-extraction
def wordShares(g, x, a) ≔ (wordCounts(g, x) + a) / safe(wordTotals(g, x, a))
multinomialScores
multinomialScores(g, x, z, a) · g one-hot membership n×K · x training counts n×p · z query counts m×p · a smoothing count scalar → log scores m×K
Let \(G\), \(X\) and \(a\) be as in wordShares, \(\theta = \operatorname{wordShares}(G, X, a)\), \(\pi = \operatorname{classPriors}(G)\), and let \(Z \in \mathbb{R}^{m \times p}\) (the argument z) be the query counts. Then
The log score \(s_{ik}\) of every query document for every class.
Intuition A document’s score is the log prior plus, for every occurrence of every word, the log probability of that word in the class. Repeated words count repeatedly and absent words count nothing, so the score is linear in the counts with weights \(\ln \theta_{kj}\).
Use it when you classify documents or other event counts (spam, topic, sentiment, support-ticket routing), especially with a large vocabulary and little training data. Longer documents give larger gaps between classes, so scores grow more decisive with length. Queries must use exactly the training vocabulary’s columns; drop words the training data never had. Use the scores to rank or combine; multinomialPredict gives the label.
text-classificationspam-filteringdocument-routing
def multinomialScores(g, x, z, a) ≔ liftedOverRows(z, ln(max(classPriors(g), 1e-300))) + einsum("ij,kj->ik", z, ln(max(wordShares(g, x, a), 1e-300)))
multinomialPredict
multinomialPredict(g, x, z, a) · g one-hot membership n×K · x training counts n×p · z query counts m×p · a smoothing count scalar → 0/1 mask m×K
Let \(G\), \(X\), \(Z\), \(a\) be as in multinomialScores and \(s = \operatorname{multinomialScores}(G, X, Z, a)\). Then
The highest-scoring class of each query document, as a one-hot row.
Intuition The class under which the document’s word counts are most probable, after weighting by the prior. The decision is linear in the counts.
Use it when you need a quick, strong baseline for text categorisation: spam filtering, topic labelling, or language identification from character n-gram counts. Train and query with the same vocabulary columns and the same a. When whether a feature appears matters more than how often (short texts, small binary vocabularies), try bernoulliPredict.
text-classificationspam-filteringbaseline-comparison
def multinomialPredict(g, x, z, a) ≔ highest(multinomialScores(g, x, z, a))
Building blocks
wordCounts(g, x)— Let \(G \in \{0,1\}^{n \times K}\) be the one-hot membership and \(X \in \mathbb{R}^{n \times p}\) the counts. Then \(\operatorname{wordCounts}(G, X)_{kj} = N_{kj} = \sum_{i=1}^{n} G_{ik} X_{ij}\). The total count of every term in every class, shape K×p.wordTotals(g, x, a)— With \(G\), \(X\), \(N\) as above and \(a \ge 0\), \(\operatorname{wordTotals}(G, X, a)_{kj} = \sum_{l=1}^{p} (N_{kl} + a) = \sum_{l} N_{kl} + p\,a\) for every \(j\). Each class’s smoothed total count, repeated across every term, shape K×p.
Bernoulli naive Bayes
The model for yes/no features. The difference from the multinomial model is that absence counts: a feature that is 0 contributes \(\ln(1 - \theta_{kj})\) rather than nothing. That makes it the better choice when the vocabulary is short and every feature is binary.
\[\theta_{kj} = \frac{N_{kj} + a}{n_k + 2a}, \qquad s_{ik} = \ln \pi_k + \sum_j \big[ z_{ij} \ln \theta_{kj} + (1 - z_{ij}) \ln(1 - \theta_{kj}) \big]\]Training data and queries must contain only 0 and 1.
presentShares
presentShares(g, x, a) · g one-hot membership n×K · x 0/1 training features n×p · a smoothing count scalar → presence probabilities K×p
Let \(G \in \{0, 1\}^{n \times K}\) (the argument g) be the one-hot class membership, with \(G_{ik} = 1\) exactly when row \(i\) is in class \(k\), let \(X \in \{0, 1\}^{n \times p}\) (the argument x) be the training features, and let \(a \ge 0\). Let \(N_{kj} = \sum_i G_{ik} X_{ij}\), \(n_k = \sum_i G_{ik}\) and \(d_k = n_k + 2a\). Then
The smoothed probability \(\theta_{kj}\) that feature \(j\) is present in a row of class \(k\).
Intuition The smoothed fraction of a class’s rows in which feature \(j\) is present. The \(+a\) over \(+2a\) pretends there were a extra rows with the feature and a without, which pulls estimates from small classes toward 1/2.
Use it when you look for the yes/no features that characterise a class, or need the parameters of Bernoulli naive Bayes. Keep a > 0 (1 is usual): with a = 0 a feature never present in a class gets probability 0, and any query that has it scores that class at the floor \(\ln 10^{-300}\), effectively ruling it out.
parameter-estimationclassification
def presentShares(g, x, a) ≔ (wordCounts(g, x) + a) / safe(einsum("kj,k->kj", ones(wordCounts(g, x)), classSizes(g) + 2 · a))
bernoulliScores
bernoulliScores(g, x, z, a) · g one-hot membership n×K · x 0/1 training features n×p · z 0/1 query features m×p · a smoothing count scalar → log scores m×K
Let \(G\), \(X\), \(a\) be as in presentShares, \(\theta = \operatorname{presentShares}(G, X, a)\), \(\pi = \operatorname{classPriors}(G)\), and let \(Z \in \{0, 1\}^{m \times p}\) (the argument z) be the queries. Then
The log joint probability \(s_{ik}\) of every query with every class.
Intuition Every feature votes, present or not: presence adds \(\ln \theta_{kj}\) and absence adds \(\ln(1 - \theta_{kj})\). So “this message does not mention an invoice” is evidence too, which the multinomial model ignores.
Use it when features are binary flags (keyword present, box ticked, symptom observed), texts are short, or the vocabulary is small. Inputs must be 0/1, so binarise counts first. With a large vocabulary the many absent words dominate the score, and multinomialScores usually does better.
text-classificationspam-filteringclassification
def bernoulliScores(g, x, z, a) ≔ liftedOverRows(z, ln(max(classPriors(g), 1e-300))) + einsum("ij,kj->ik", z, ln(max(presentShares(g, x, a), 1e-300))) + einsum("ij,kj->ik", 1 − z, ln(max(1 − presentShares(g, x, a), 1e-300)))
bernoulliPredict
bernoulliPredict(g, x, z, a) · g one-hot membership n×K · x 0/1 training features n×p · z 0/1 query features m×p · a smoothing count scalar → 0/1 mask m×K
Let \(G\), \(X\), \(Z\), \(a\) be as in bernoulliScores and \(s = \operatorname{bernoulliScores}(G, X, Z, a)\). Then
The highest-scoring class of each query, as a one-hot row.
Intuition The class whose pattern of present and absent features best matches the query, after weighting by the prior.
Use it when you classify from yes/no features: symptom checklists, survey answers, short messages flagged by keywords. The output is a one-hot mask, and ties mark every tied class. For continuous measurements use gaussianPredict; for word or event counts use multinomialPredict.
classificationspam-filteringsurvey-analysis
def bernoulliPredict(g, x, z, a) ≔ highest(bernoulliScores(g, x, z, a))
What is not here
Quadratic discriminant analysis (QDA), which drops the independence assumption and gives each class its own full covariance matrix, is not provided: it needs one p×p inverse per class. Linear discriminant analysis, which shares one covariance matrix across classes, is available in Prelude.Multivariate.
Examples
The examples share one file, Classifiers.hb. Inputs are passed as Tensor, because every result has a different shape from its inputs.
open Prelude.Bayes
fn gaussianClass(g, x, z) ↦ gaussianPredict(g, x, z)
fn gaussianChances(g, x, z) ↦ gaussianPosterior(g, x, z)
fn gaussianCentres(g, x) ↦ classMeans(g, x)
fn gaussianSpreads(g, x) ↦ classVariances(g, x)
fn countScores(g, x, z, a) ↦ multinomialScores(g, x, z, a)
fn countClass(g, x, z, a) ↦ multinomialPredict(g, x, z, a)
fn presenceClass(g, x, z, a) ↦ bernoulliPredict(g, x, z, a)
fn presenceScores(g, x, z, a) ↦ bernoulliScores(g, x, z, a)
Classify measurements with Gaussian naive Bayes
One feature, two classes: {1, 2, 3} and {7, 8, 9}. The class means are 2 and 8 and both variances are \(2/3\). For the query 4.5 the log-odds in favour of the first class are \(\big((4.5-8)^2 - (4.5-2)^2\big) / (2 \cdot 2/3) = 4.5\), so its posterior is \(1/(1 + e^{-4.5}) = 0.989013\).
using MLambda.Hilbert.Runtime;
using Examples;
var g = (Tensor)new double[,] { { 1, 0 }, { 1, 0 }, { 1, 0 }, { 0, 1 }, { 0, 1 }, { 0, 1 } };
var x = (Tensor)new double[,] { { 1 }, { 2 }, { 3 }, { 7 }, { 8 }, { 9 } };
var z = (Tensor)new double[,] { { 2.5 }, { 4.5 }, { 8.5 } };
var means = Classifiers.GaussianCentres(g, x);
var variances = Classifiers.GaussianSpreads(g, x);
var predicted = Classifiers.GaussianClass(g, x, z);
var posterior = Classifiers.GaussianChances(g, x, z);
Output (verified, six decimals; matrices listed row by row):
means = [[2], [8]]
variances = [[0.666667], [0.666667]]
predicted = [[1, 0], [1, 0], [0, 1]]
posterior = [[1, 0], [0.989013, 0.010987], [0, 1]]
Classify short documents by word counts
A vocabulary of three words and four training documents, two per class. The first class uses mostly word 1; the second mostly words 2 and 3. With smoothing \(a = 1\) the first class’s word probabilities are \((6, 1, 2)/9\), so a query containing word 1 once scores \(\ln 0.5 + \ln(6/9) = -1.098612\) for that class.
var g = (Tensor)new double[,] { { 1, 0 }, { 1, 0 }, { 0, 1 }, { 0, 1 } };
var counts = (Tensor)new double[,] { { 3, 0, 1 }, { 2, 0, 0 }, { 0, 3, 1 }, { 0, 2, 2 } };
var docs = (Tensor)new double[,] { { 1, 0, 0 }, { 0, 1, 1 } };
var scores = Classifiers.CountScores(g, counts, docs, 1.0);
var predicted = Classifiers.CountClass(g, counts, docs, 1.0);
Output (verified, six decimals):
scores = [[-1.098612, -3.091042], [-4.394449, -2.310884]]
predicted = [[1, 0], [0, 1]]
Classify yes/no features with Bernoulli naive Bayes
The same classes with binary features. For the first class the smoothed presence probabilities are \((3, 1, 2)/4\), so the query \((1, 0, 0)\) scores \(\ln 0.5 + \ln 0.75 + \ln 0.75 + \ln 0.5 = -1.961659\): the two absent features contribute as well.
var g = (Tensor)new double[,] { { 1, 0 }, { 1, 0 }, { 0, 1 }, { 0, 1 } };
var seen = (Tensor)new double[,] { { 1, 0, 1 }, { 1, 0, 0 }, { 0, 1, 1 }, { 0, 1, 1 } };
var asks = (Tensor)new double[,] { { 1, 0, 0 }, { 0, 0, 1 } };
var scores = Classifiers.PresenceScores(g, seen, asks, 1.0);
var predicted = Classifiers.PresenceClass(g, seen, asks, 1.0);
Output (verified, six decimals):
scores = [[-1.961659, -4.85203], [-3.060271, -2.654806]]
predicted = [[1, 0], [0, 1]]