Frequency of appearance of letters of the Russian alphabet. Frequency of letters in Russian

Do you know that some letters of the alphabet are found in words more often than others... Moreover, the frequency of use of vowels in the language is higher than consonants.

Which letters of the Russian alphabet are most often or least often found in words used to write text?

Statistics deals with the identification and study of general patterns. With the help of this scientific direction, you can answer the question posed above by counting the number of each letter of the Russian alphabet, the words used, and selecting an excerpt from the works of various authors. For their own interest and for the sake of something to do out of boredom, everyone can do this on their own. I will refer to the statistics of an already conducted study...

Russian alphabet Cyrillic. During its existence, it experienced several reforms, as a result of which the modern Russian alphabet system, including 33 letters, was formed.

o — 9.28%
a — 8.66%
e — 8.10%
and - 7.45%
n — 6.35%
t — 6.30%
p — 5.53%
s — 5.45%
l - 4.32%
in — 4.19%
k — 3.47%
n — 3.35%
m — 3.29%
y - 2.90%
d — 2.56%
I - 2.22%
s — 2.11%
b — 1.90%
z — 1.81%
b — 1.51%
g — 1.41%
th — 1.31%
h — 1.27%
yu — 1.03%
x — 0.92%
f — 0.78%
w — 0.77%
c — 0.52%
sch — 0.49%
f — 0.40%
e — 0.17%
ъ — 0.04%

The Russian letter with the highest frequency of use is the vowel “ ABOUT", as has already been rightly suggested here. There are also typical examples like “ DEFENSE"(7 pieces in one word and nothing exotic or surprising; very common for the Russian language). The high popularity of the letter “O” is largely explained by such a grammatical phenomenon as full vowel. That is, “cold” instead of “cold” and “frost” instead of “scum”.

And at the very beginning of words, the consonant letter “” is most often found P" This leadership is also confident and unconditional. Most likely, the explanation is provided by a large number of prefixes starting with the letter “P”: pere-, pre-, pre-, pri-, pro- and others.

The frequency of use of letters is the basis of cryptanalysis.

In general, there is such a topic - frequency analysis of text. It is argued that for a given language the frequency of occurrence of individual letters in a meaningful text is a stable value. Combinations of two, three (digrams, trigrams) and four letters are also stable.
This fact, in particular, was used in cryptography to break ciphers.

I'm not very good at cryptography, and the only thing that comes to mind is breaking a direct substitution cipher. It must be said that the most primitive cipher is when the characters of the original alphabet used in the message are converted into other characters according to a certain rule. Such ciphers, by the way, could be opened without the use of statistical analysis (where, to reduce the error, the presence of rather large pieces of text is obviously required), but simply by guessing about some words - see the story “The Dancing Men”.

And the final touch (optional). Sometimes (almost always for now) the calculator needs to be provided with a description - what the parameters are, what formulas it uses, and in general, why it’s all for - just like I’m doing now. To do this, an article is written, and the calculator is inserted directly into the article. To write an article, select the menu item “Create...” -> “Article” on the main page of the “My Calculators” section and start writing. To insert a calculator, press the button with the large underlined letter A and select the newly created calculator in the dialog that opens.

Which letter is most often used in Russian? And what is the easiest way to find out and got the best answer

Answer from Viktor Edinovich[guru]

The only correct answer! At one time, any typographer knew from experience which font (letter) was used the most when typing.
Victor Edinovich
Enlightened
(38377)
No. I'm just a former encyclopedist...

Answer from Elizaveta Ventsevich[guru]
I think "A".

Answer from Alexandra Lapikova[guru]
according to the degree of wear on the writer’s keyboard

Answer from YolaFka[guru]
bukaFka AAAAAAAAAAAA))))

Answer from BOBA BOBAHOB[guru]
I think that the letter "X" is evidence? just take a walk down the street

Answer from * [expert]
the most commonly used 3 letters are U Y X

Answer from On[guru]
probably Kommersant

Answer from Dyusmikeev Valery[guru]
The letter a

Answer from &Ъ[guru]
How everyone jokes. Of course, the letter O!!!

Answer from Pavel Makagonov[guru]
O, then E, then I, then A. Then the consonants N, T, S, R, etc.
I had to do the math based on a large number of texts. Don't trust the hooligans.

Answer from Elena Strathberry[active]
Of course the letter R. Because it is in the middle of the keyboard!! ! 🙂

Answer from Inga zajontz[guru]
we take any dictionary, most of all words starting with “p”, after with “r”, after with “o”, after with “T”, in the end the most characteristic word for Russian is spaciousness and simplicity
according to Dahl

Answer from Alexander Reiser[guru]
Try to guess which letter of the Russian language is most common?
Letter O. Frequency of occurrence – 0.090. That is, in a sequence of 1000 letters, the letter O will appear on average 90 times.
Which letters occupy leading positions and are found most often?
Vowels. They are the ones that are most often found in the Russian language, making our speech “melodious”. Following O are the letters E and E (together, they are not separated, because when writing, they often lose E). Occurrence frequency – 0.072. Behind them are the letters A and I. The frequency of appearance of each of them is 0.062.
Is this enough to decode text? In principle, yes, if we are sure that the sequence of codes contains text in Russian in some of the encodings known or unknown to us. But there is always some probability that the presented sequence of codes has nothing to do with the Russian language. To verify this, you need to use a frequency dictionary of words in the Russian language.
What word do you think occurs most often in the Russian language?
The conjunction I is followed by equally short parts of speech - IN, NOT, OH, ON. But most often, of course, there will be a gap.
The decoded text must be checked for a match using the frequency dictionary. And only with more or less complete coincidence can we say that we are dealing with a text written in Russian.
This gives 100% results if the text is large enough. If we are talking about several words, then the frequency of letters (and even more so words) may be disrupted. What to do in this case? We need to offer the reader several options to choose from. Decode the text using one of the most common vowel letters. Vowels! They will always appear more often than other letters. If the text is encoded in Russian, then success is guaranteed.

In this article we will begin a discussion of an extremely interesting topic - the use of statistics for the analysis of text information. Note that the use of statistics for text analysis is a traditional task.

First, we will present some interesting facts regarding the frequency of occurrence of letters and their combinations in different languages ​​(for more details, see the book). In subsequent articles we will show how to apply more complex methods of analysis and graphical representation.

Frequency characteristics of text messages

So, the text consists of words, words of letters. The number of different letters in each language is limited and the letters can simply be listed. Important characteristics of the text are the repetition of letters, pairs of letters (digrams) and in general m-OK ( m-gram), compatibility of letters with each other, alternation of vowels and consonants and some others. It is remarkable that these characteristics are quite stable. We leave the question “why” behind the scenes.

Using the system STATISTICA You can check these patterns, for example, in Internet texts.

The idea is to count the number of occurrences of each n m possible m-gram in sufficiently long plaintexts T=t 1 t 2 …t l, made up of letters of the alphabet ( a 1 , a 2 , ..., a n). At the same time, consecutive m-grams of text:

t 1 t 2 ...t m , t 2 t 3 ... t m+1 , ..., t i-m+1 t l-m+2 ...t l.

If – number of occurrences m-grams a i1 a i2 ...a im in the text T, A L– total number of counted m-gram, then experience shows that for sufficiently large L frequencies

for this m-grams differ little from each other.

Because of this, the relative frequency (1) is considered to be an approximation of the probability P (a i1 a i2 ...a im) appearance of this m-grams in a randomly selected place in the text (this approach is adopted in the statistical determination of probability).

Below are tables of letter frequencies (in percentage) for a number of European languages. Data taken from the book.

Some difference in frequency values ​​in the tables given in various sources is explained by the fact that frequencies significantly depend not only on the length of the text, but also on its nature. For example, in technical texts the rare letter F can become quite common due to the frequent use of words such as function, differential, diffusion, coefficient, etc.

Even greater deviations from the norm in the frequency of use of individual letters are observed in some works of art, especially in poetry. Therefore, to reliably determine the average frequency of letters, it is desirable to have a set of different texts borrowed from various sources. However, as a rule, such deviations are insignificant, and to a first approximation they can be neglected.

A visual representation of the frequencies of letters is given by the occurrence diagram. So, for the English language, in accordance with the table, such a diagram is shown in Fig. 1. To build it, we used the system STATISTICA.

For the Russian language, the frequencies (in descending order) of the characters of the alphabet in which they are identified E c Yo, b With Kommersant, and there is also a space sign (-) between words, are given in the following table (see).

Based on the table, we obtain the following frequency diagram (Fig. 2).

There is a mnemonic rule for memorizing the ten most common letters of the Russian alphabet. These letters make up the ridiculous word HAY. You can also suggest a similar way of memorizing common letters in the English language, for example, using the word TETRIS-HONDA (see table).

The frequency characteristics of bigrams, trigrams and fourgrams of meaningful texts are also stable.

We present tables of bigram frequencies for the Russian and English languages ​​(the tables are borrowed from the book). For convenience, they are divided into four parts according to the following scheme:

Nice tables k-gram is easy to obtain using the texts of electronic versions of many books contained on CDs.

To obtain more accurate information about plaintexts, you can build and analyze tables k-gram at k>2, but for educational purposes it is enough to limit ourselves to bigrams. Unevenness k-gram (and even words) is closely related to a characteristic feature of open text - the presence in it of a large number of repetitions of individual text fragments: roots, endings, suffixes, words and phrases. So, for the Russian language, such familiar fragments are the most common bigrams and trigrams:

ST, BUT, EN, TO, NA, OV, NI, RA, VO, KO
STO, ENO, NOV, TOV, OVO, OVA

Information about the compatibility of letters, that is, about the preferred connections of letters with each other, is useful, which can be easily obtained from bigram frequency tables.

This refers to a table in which the most preferred “neighbors” are located to the left and right of each letter (in descending order of frequency of the corresponding bigrams). Such tables usually also indicate the proportion of vowels and consonants (in percentage) preceding (or following) a given letter.

Combination of Russian letters:

When analyzing the compatibility of letters with each other, one should keep in mind the dependence of the appearance of letters in plain text on a significant number of preceding letters. To analyze these patterns, the concept of conditional probability is used.

Observations of plaintexts show that the following inequalities hold for conditional probabilities: p(a i1)≠p(a i1 /a i2), p(a i1 /a i2)≠p(a i1 /a i2 a i3),....

The question of the dependence of the letters of the alphabet in plain text on previous letters was systematically studied by the famous Russian mathematician A. A. Markov (1856 – 1922). He proved that the occurrences of letters in plaintext cannot be considered independent of each other. In this regard, A. A. Markov noted another stable pattern of open texts associated with the alternation of vowels and consonants. He calculated the frequency of occurrence of vowel-vowel bigrams ( G, G), vowel-consonant ( G, With), consonant-vowel ( With, G), consonant-consonant ( With, With) in Russian text with a length of 10 5 characters. The calculation results are shown in the following table:

From this table it can be seen that the Russian language is characterized by alternation of vowels and consonants, and the relative frequencies can serve as approximations of the corresponding conditional and unconditional probabilities:

p(G/With)≈0.663, p(With/G)≈0.872,
p(G)≈0.432, p(With)≈0.568.

After A. A. Markov, the dependence of the appearance of letters in a text, following several previous ones, was studied by methods of information theory by K. Shannon. In fact, they showed, in particular, that such a dependence is noticeable to a depth of approximately 30 characters, after which it is practically absent.

Proportion of vowels in a literary text:

The above patterns apply to ordinary “readable” plaintexts used in human communication. As noted earlier, these patterns play a big role in cryptanalysis. In particular, they are used in the construction of formalized criteria for plaintext, which make it possible to apply methods of mathematical statistics in the problem of plaintext recognition in a message stream. When using special alphabets, similar studies of the frequency characteristics of “open texts” that arise, for example, during machine-to-machine information exchange or in data transmission systems are required. In these cases, constructing formalized criteria for “clear text” is a much more difficult task.

As an example, we give the frequency characteristics of the letters of the English alphabet that are part of the ASCII code.

In addition to cryptography, the frequency characteristics of clear messages are significantly used in other areas. For example, a computer keyboard, a typewriter or a Linotype is a wonderful embodiment of the idea of ​​speeding up typing, associated with optimizing the arrangement of the letters of the alphabet relative to each other, depending on the frequency of their use.

Literature:

Alferov A.P. et al., "Cryptography"

Yaglom A.M., Yaglom I.M., Probability and information, M.: Nauka, 1973.

Baudouin C., Elements de cryptographie / Ed. Pedone A. – Paris, 1939.

Friedman W. F., Callimahos D., Military cryptanalysis, Part i, Vol 2, Aegean Park Press, Laguna Hills CA, 1920.

The method proposed by Al-Kindi is easier to explain from the point of view of the Russian alphabet. First of all, it is necessary to study a sufficiently long passage of text in Russian, or several passages of different texts, in order to establish the frequency of occurrence of each letter of the alphabet. In russian language O- the most common letter, after it e, then A and so on, as indicated in the table. Then we will study the ciphertext and determine the frequency of occurrence of each character in it. For example, if the most frequent character in the ciphertext YU, then most likely it should be replaced with the letter O. If the second most common character in the ciphertext E, then it should probably be replaced with e, and so on. Thanks to Al-Kindi's method, known as frequency cryptanalysis, there is no need to check each of the billions of potential keys. Instead, you can decrypt a message simply by analyzing the frequency of the characters in it.

However, frequency cryptanalysis does not completely solve the problem of breaking monoalphabetic ciphers. Its applicability depends on the size and nature of the text. The average letter frequencies of any language will not always correspond to the letter frequencies of a particular text. For example, a short message discussing the influence of the atmosphere on the movement of zebras in Africa, “Because of the ozone holes from Zanzibar to Zambia and Zaire, zebras run in zigzags,” if encrypted with a monoalphabetic cipher, could not be decrypted using simple frequency cryptanalysis. Since the letter h in this message it occurs an order of magnitude more often than in simple speech. A rare letter in technical texts f can become quite common due to the frequent use of words such as function, differential, diffusion, coefficient, etc.

If it is not possible to decipher a cryptogram using simple frequency cryptanalysis (for example, if the message is too short), Al-Kindi suggests using characteristic combinations of letters or, conversely, the incompatibility of certain letters with each other. For example, the most common bigrams (groups of two letters) of the Russian language: st, But, en, That, on, ov, neither, ra, in, co. Statistics on the combination of vowels and consonants are important. For example, before letters b, s, ъ and after uh vowels cannot appear, and any vowel is followed by a consonant with a probability of 87%. Also a clue for a cryptanalyst can be the generally accepted introductory words that are used in almost every language. For example, in Arabic, “In the name of God, the merciful and merciful” (بسم الله الرحمن الرحيم) was often used. When deciphering poems, you can use rhymes and feet.

Arabic letters: their order and repetition

Al-Kindi provides a table with the frequencies of letters of the Arabic alphabet calculated in a sample of seven sheets of text.

There are 28 letters in the Arabic alphabet. Of these, 27 can denote consonant sounds, 3 (ﺍ (/aː/), ﻭ (/uː/), ﻱ (/iː/)) - long vowel sounds, there are no letters denoting short vowels (for example, in the word Muhammad they are written only four consonants: محمد). Thus, purely consonant letters predominate in Arabic writing. However, this fact does not contradict the statement indicated at the beginning of the treatise that the most common letter in the writing of any language is, as a rule, a vowel, since in Arabic it is ﺍ (/aː/).