Conditional entropy

This online calculator calculates entropy of Y random variable conditioned on X random variable and X random variable conditioned on Y random variable given a joint distribution table (X, Y) ~ p

This page exists due to the efforts of the following people:

Timur

Calculator : Conditional entropy - Author

Created: 6 years ago, Last updated: 5 years ago

Articles that describe this calculator

Conditional entropy

Conditional entropy

Joint distribution (X,Y) ~ p

Show details

Calculation precision

Digits after the decimal point: 2

H(Y|X)

1.09

H(X|Y)

0.68

The file is very large. Browser slowdown may occur during loading and creation.

Calculation Details

Step
$P_{(x,y)} = \begin{bmatrix}p(x_1,y_1)&p(x_1,y_2)&p(x_1,y_3)&\\p(x_2,y_1)&p(x_2,y_2)&p(x_2,y_3)&\\p(x_3,y_1)&p(x_3,y_2)&p(x_3,y_3)&\\\end{bmatrix} = \begin{bmatrix}0.1&0&0&\\0.2&0.3&0.2&\\0&0&0.2&\\\end{bmatrix}$
$p_x = \begin{bmatrix}0.10\\0.70\\0.20\\\end{bmatrix}$
$p_{y_j\|x_i} = \begin{bmatrix}1&0&0&\\\frac{0.2}{0.70}&\frac{0.3}{0.70}&\frac{0.2}{0.70}&\\0&0&1&\\\end{bmatrix}$
$H(Y\|X) = -\Big(0.1\cdot log_21+0\cdot log_20+0\cdot log_20+0.2\cdot log_2\frac{0.2}{0.70}+0.3\cdot log_2\frac{0.3}{0.70}+0.2\cdot log_2\frac{0.2}{0.70}+0\cdot log_20+0\cdot log_20+0.2\cdot log_21\Big)$
$p_y = \begin{bmatrix}0.30\\0.30\\0.40\\\end{bmatrix}$
$p_{x_j\|y_i} = \begin{bmatrix}\frac{0.1}{0.30}&0&0&\\\frac{0.2}{0.30}&1&\frac{0.2}{0.40}&\\0&0&\frac{0.2}{0.40}&\\\end{bmatrix}$
$H(X\|Y) = -\Big(0.1\cdot log_2\frac{0.1}{0.30}+0.2\cdot log_2\frac{0.2}{0.30}+0\cdot log_20+0\cdot log_20+0.3\cdot log_21+0\cdot log_20+0\cdot log_20+0.2\cdot log_2\frac{0.2}{0.40}+0.2\cdot log_2\frac{0.2}{0.40}\Big)$
Items per page: 1 - 7 of 7

URL copied to clipboard

Similar calculators

#entropy #information conditional entropy entropy information theory probability

PLANETCALC Online calculators

Conditional entropy

This page exists due to the efforts of the following people:

Timur

Articles that describe this calculator

Conditional entropy

Calculation Details

Similar calculators

Comments

PLANETCALC Online calculators

Articles that describe this calculator

Conditional entropy

Similar calculators

Comments

Share this page