Hartley formula

The Hartley formula determines the amount of information contained in a message of length n .

There is an alphabet A, from the letters of which the message is composed:

|A|=m.

{\ displaystyle | A | = m.}

{\ displaystyle | A | = m.}

The number of possible options for different messages:

N=m^{n},

{\ displaystyle N = m ^ {n},}

{\ displaystyle N = m ^ {n},}

where N is the possible number of different messages, m is the number of letters in the alphabet, n is the number of letters in the message.

Example: The alphabet consists of two letters “B” and “X”, the message length is 3 letters - thus, m = 2, n = 3. With the alphabet chosen by us and the message length, you can make $N=m^{n}=2^{3}=8$ ${\ displaystyle N = m ^ {n} = 2 ^ {3} = 8}$ ${\ displaystyle N = m ^ {n} = 2 ^ {3} = 8}$ different messages: “BBB”, “BBX”, “BXB”, “BXX”, “XBB”, “XBX”, “XXB”, “XXX” - there are no other options.

The Hartley formula is determined by:

I=\log _{2}N=n\log _{2}m,

{\ displaystyle I = \ log _ {2} N = n \ log _ {2} m,}

{\ displaystyle I = \ log _ {2} N = n \ log _ {2} m,}

where I is the amount of information in bits .

With equiprobable characters $p={\frac {1}{m}},m={\frac {1}{p}}$ ${\ displaystyle p = {\ frac {1} {m}}, m = {\ frac {1} {p}}}$ ${\ displaystyle p = {\ frac {1} {m}}, m = {\ frac {1} {p}}}$ Hartley's formula goes into proprietary information .

The Hartley formula was proposed by Ralph Hartley in 1928 as one of the scientific approaches to evaluating reports.

Illustration

Suppose we need to find or define something in a particular system. There is such a search method as “ halving ”. For example, someone guesses a number from 1 to 100, and the other must guess it, receiving only the answer “yes” or “no”. The question is asked: "is the number less than N ?". Any of the answers “yes” and “no” will halve the search area. Further, according to the same scheme, the range is again divided in half. Ultimately, the hidden number will be found.

How many questions must be asked in order to find a conceived number from 1 to 100. Suppose, a hidden number 27. Variant of dialogue:

  More than 50?  Not.
 More than 25?  Yes.
 More than 38?  Not.
 Less than 32?  Yes.
 Less than 29?  Yes.
 Less than 27?  Not.
 Is it the number 28?  Not.

If the number is not 28 and not less than 27, then this is clearly 27. In order to guess the number from 1 to 100 using the “halving” method, we needed 7 questions.

You can just ask: is this the number 1? Is it the number 2? And so on. But then you need a lot more questions. “Halving” is the optimal way to find the number in this case. The amount of information embedded in the answer “yes” / “no”, if these answers are equally probable, is equal to one bit (really, because the bit has two states: 1 or 0). So, to guess the number from 1 to 100, it took us seven bits (seven answers “yes” / “no”).

N=2^{k}

{\ displaystyle N = 2 ^ {k}}

Such a formula can represent how many questions (bits of information) will be required to determine one of the possible values. N is the number of values, and k is the number of bits. For example, in our example, 27 is less than 28, but more than 26. Yes, we might need only 6 questions if the number we had requested was 28.

Hartley Formula:

k=\log _{2}N.

{\ displaystyle k = \ log _ {2} N.}

The amount of information ( k ) needed to determine a specific element is the base 2 logarithm of the total number of elements ( N ).

Shannon's Formula ^[1]

When events are not equally probable, Shannon's formula can be used:

I=-\sum _{i}p_{i}\log _{2}p_{i},

{\ displaystyle I = - \ sum _ {i} p_ {i} \ log _ {2} p_ {i},}

where p _{i is the} probability of the ith event.

Hartley formula

Illustration

Shannon's Formula ^[1]

See also

More articles:

Hartley formula

Illustration

Shannon's Formula [1]

See also

More articles:

Shannon's Formula ^[1]