The International College of Economics and Finance (ICEF) in Moscow was established in 1997 jointly by the London School of Economics and Political Science (LSE) in London and the Higher School of Economics (HSE) in Moscow. The college offers a 4-year bachelor's program, which is considered to be one of the top programs in economics in Russia. Each year about 200 students enter the program, typically immediately after high school. In their first year the students follow, among other subjects, a course called Statistics-1, and in their second year they follow Statistics-2. Both courses are compulsory. Our data are obtained from students following Statistics-2 over a 5-year period, 2011–2015. In total, 964 students took this course during these 5 years.

In Statistics-2 students take three exams every year, at the end of October (exam 1), the end of December (exam 2), and the end of March (exam 3). The exams are written exams, not multiple choice, and each consists of two parts (80 min each) with a 10 min break between the two parts. The level of the exam questions is the same in the two parts. In order to avoid cheating, students are not allowed to leave and come back during each part of the exam. At the end of part 1 and at the end of the exam the examiner collects each student's work. Each part is graded out of 50 points.

In addition, students have weekly homework assignments although these are not compulsory. All handed-in assignments are graded (out of 100). The variable HW denotes for each student the sum of all assignments' grades divided by the total number of assignments. For example, if a student hands in 20 of the 25 assignments and scores 100 (the maximum) for each, then HW = (20·100)/25 = 80.

After completion of the three course exams, students take two additional exams (some only take one) in early May administered by the University of London, called STAT₁ and STAT₂. These, like the other exams, are also graded with a maximum score of 100. The final grade G_tot for the course is then determined as

where G_j is the grade obtained in exam j.

Students fail if G_tot is smaller than some threshold to be determined by the teacher, but lying between 32 and 37. Student also fail if G₃, the grade in the third exam, is <25. Exam 3 thus plays a special role in two ways: its weight is higher than for the other two exams and there is a threshold grade of 25.

At the end of the first part of each of the three exams each student was asked to forecast (out of 100) his/her grade for this exam (the two parts together). Students were told that answering this question is voluntary: they can answer or they can skip the question. They were also told that their answers could be used for research purposes anonymously. At the moment when the student writes down the forecast he/she knows the questions and his/her answers in part 1, but the student does not yet know the questions of part 2. To encourage students to fill in their forecast and to actually try their best, a bonus is promised. If the difference between the forecast and the grade is ≤ 3 in absolute value, then one bonus point is added to the grade. For example, if the forecast is 49 and the grade is 52, then the grade for this exam is marked up to 53. This procedure had to be and has been approved by the ICEF administration. As a result of the procedure and the possibility of a bonus, the response rate was extremely high (97%). The idea of giving each student an incentive to express his/her opinion was also used in a recent experiment by Blackwell (2010), where students were asked to assess the difficulty of an assignment by guessing the class average, earning a bonus if their guess was close enough.

Smart (or risk averse) students utilize this bonus in the third exam, where the grade must be ≥25. If the student chooses the forecast F₃ such that 21 ≤ F₃ ≤ 27, then a grade G₃ = 24 would be marked up to 25. Some students actually do this, but they then typically choose F₃ = 24 or 25 and not, say, 21 or 27. The special role of the third exam and the overrepresentation of 24 and 25 in the sample of third exam forecasts have to be taken into account when we do our statistical analysis, and we shall discuss this issue further below.

The data consist of the grades G_j and the forecasts F_j (j = 1, 2, 3) for each of our students, and our interest is focused on the excess expectation in exam j:

We have some background knowledge on each student, namely the grades of the first-year calculus (calc) and first-year statistics (stats) exams, the grade point average at the end of the first year (gpa), and whether the student is male or female (female = 1 for women and 0 for men). We also know in which year the exam took place (year = 2011, 2012, 2013, 2014, 2015).

For the homework assignments we know for each student how many assignments the student handed in (nhw_j), the sum of the grades per exam period (shw_j), and the number of handed-out assignments (nhwmax_j). The index j now refers to a period rather than to a point in time: j = 1 refers to the period up to the first exam, j = 2 to the period between the first and second exams, and j = 3 to the period between the second and third exams. The number of handed-out assignments (nhwmax_j) may vary from year to year, in fact from 4 to 7 in period 1, from 6 to 7 in period 2, and from 9 to 12 in period 3.

From these “raw” data we can compute the ratios

which denote, respectively, the relative number of submitted assignments for each student in period j (0 ≤ rnhw_j ≤ 1), and the average submitted assignment grade for each student in period j (0 ≤ rshw_j ≤ 100), where we set rshw_j = 0 if nhw_j = 0. These ratios will be used later in the analysis.

In order to obtain a clean and complete sample, some data screening was necessary. Of the original 964 students we excluded those students who (a) did not take all three exams; or (b) had repeated the first year; or (c) had failed the course last year; or (d) had taken a break between the first and second year. This left us with 840 students.

Of these 840 students, a further 248 were excluded because they did not provide all three forecasts or we didn't have their first-year results. As a consequence, 592 students remain on which we have complete information. The results of the University of London exams are not used in our analysis.

A summary of these “raw” and basic data is provided in Table 1 (Kernel density plots of the basic data are provided as Supplementary Material). There is substantial variation in the exam grades G and the forecasts F over the years, possibly because the difficulty of the exams varies (although the same instructor taught the course over this period) or the quality of the student population varies (because of changes in admission policies). This suggests that year dummies may be important. On the whole it seems that students are too optimistic about their abilities, because F_j > G_j occurs more frequently than F_j < G_j. Also, in the first two exams large deviations occur where |F_j − G_j| can be larger than 10, in contrast to the third exam where the deviation is much smaller. This suggests that students learn from their past forecast errors.

From these raw data it is not immediately clear what the answers are to our questions. To achieve this we need more sophisticated statistical techniques than simple averages. In the next section we will address each of our research questions in turn and develop the required models as we progress.

Our first question is whether our students have rational expectations about their exam grades. In the Introduction we mentioned some literature where it is found that students overestimate their abilities, that is, that they are not rational. If we also find this (as we shall) then a second question arises, namely whether male and female students are equally irrational or that perhaps female students behave more rationally than male students.

Our experimental data differ from the data in most papers in three respects: first, we use a 0–100 grade system, while other papers typically use the more discrete F-D-C-B-A (0–4) grade system; second, our students make their forecast after they have already finished half the exam; and third, our students have a real incentive to make their forecast as precise as possible, as they get a bonus for an accurate forecast.

There is, in addition, one other feature of our data, namely the fact that we collected exam results and the associated forecasts during 5 years (2011–2015). We know from the previous section that the exams are not equally difficult in each year, and these discrepancies need to be taken into account. Thus, following Hossain and Tsigaris (2015), we regress the excess expectation D_{j, i} = F_{j, i} − G_{j, i} for student i in exam j for each of the three exams separately, and include year dummies year12_i − year15_i. The regression then reads

where x_{j, i} is the vector of all available information at the time of exam j of the i-th student's previous academic achievements, and the control variables are a vector of time dummies

and the female/male dummy female_i. The α_j, β_j, γ_j, and δ_j are unknown parameters (parameter vectors), and ε_{j, i} is the random error, which we assume to be independently and identically distributed with mean zero.

We define a student to be rational when the conditional expectation E(D_{j, i}|x_{j, i}) = 0 and this translates to testing the null hypothesis

for each of the three exams j = 1, 2, 3. Note that the dimension of β_j is not the same for each j, because more information is available at the second exam than at the first exam, and even more information is available at the third exam. In fact, the dimension is 5 at the first exam (calc, stats, gpa, rnhw₁, rshw₁), 8 at the second (the previous plus G₁, rnhw₂, rshw₂), and 11 at the third (the previous plus G₂, rnhw₃, rshw₃).

Recall that the third exam is special because students fail the course if G₃ < 25. From the student's point of view it makes good sense to predict 21 ≤ F₃ ≤ 27, because then (and only then) a grade G₃ = 24 will be marked up to 25. This is “rational” behavior, but not according to our definition. In practice, these students choose G₃ = 24 or 25, but almost never 26 or 27. Also, some students are confused and believe that this rule applies to all three exams and not only to the third. To avoid these problems we only include forecasts which satisfy F_{j, i} > 25. We recognize that this censoring could (slightly) shift the data so that finding overconfidence becomes more likely. Since this is something that the data will not reveal, we need to assume that such a shift does not take place.

Table 2 contains the regression results for each of the three exams separately. The last row in the table contains the p-value of the F-test used for testing the hypothesis that β_j = 0. The reported p-values are < 0.2% thus rationality is firmly rejected in this model. The p-value is lowest at the first exam, still very low at the second (where more information is available), and much higher (but still < 0.2%) at the third (where even more information is available). More information thus leads to more rational decisions, which is not as obvious as it may seem, because too much information might turn into confusion and lead to less rational behavior. This, however, does not happen here.

Some preliminary conclusions can be drawn from Table 2. First, it seems that good students (high marks in calc, stats, and gpa in the previous year) are more cautious than not so good students in their predictions, at least for the first exam. If the student does well in the first exam, then he/she becomes less cautious and in fact tends to overpredict the results of both the second and the third exam. Having learnt their lesson, students become more cautious again: doing well in the second exam leads to more rather than to less caution in predicting their mark for the third exam. There is a big impact of gender. Women are more cautious than men in all three exams, although the impact diminishes over time.

Kruger and Dunning (1999) concluded that low-performing students significantly overestimate their performance, while high-performing students are more accurate in their forecasts. Feld et al. (2017) pointed out that this effect could at least partially be explained by measurement errors. They showed that after correction for measurement errors (using instrumental variables) the Dunning-Kruger effect is still observed, but significantly weaker than before the correction.

To further investigate the difference in rationality between women and men, we also estimate the extended model.

for men and women separately. The results are presented in Table 3, where we have again excluded all students with F_{j, i} ≤ 25.

By including the female dummy (as in Table 2) we distinguish between men and women, but only by allowing the level to change from α_j for men to α_j + δ_j for women. By separating men and women (as in Table 3, now of course without the female dummy), we also allow the β_j-coefficients to be different.

Our preliminary conclusions still hold in this extended framework. Women are more cautious than men. Good students are more cautious than not-so-good students in their predictions, at least for the first exam. If the student does well in the first exam, then he/she becomes too optimistic in predicting the second exam, but doing well in the second exam does not lead to such optimism. The p-values are higher than in Table 2 but still well under 0.1%, except for women in the third exam where the p-value is close to 5%. We thus find that our female students became more rational in the third exam, while men continue to exhibit irrational behavior.

It is often thought that women behave more rationally than men, and this is indeed what we find. But there is no consensus in the literature. Ballard and Johnson (2005) reported that gender is a significant determinant of student expectations: women in an introductory microeconomics course expected a grade that was one-fourth of a letter grade (0.25 on a 4.0 scale) lower than the grade expected by the men. However, after controlling for expectations and secondary-schooling experience with economics, the gender effect became small and insignificant. Hossain and Tsigaris (2015) also found that gender makes no difference in this respect.

In the previous section we rejected rationality in predicting exam results and we saw that there is a difference between male and female students. Our next step is to try and explain this lack of rationality, and our hypothesis is that students (especially male students) are too confident about their abilities. When a student has more confidence than is justified by his or her grades, we call this student “overconfident”; see i.e., Windschitl and O'Rourke Stuart (2015).

It makes sense that a student who does well in exams gains in confidence. But perhaps the opposite is also true, that is, a confident student-other things being equal-performs better than one lacking in confidence (Ballard and Johnson, 2005; Johnson and Fowler, 2011). In addition to studying overconfidence we also try to answer this somewhat subtler question in the current section.

An overconfident student will produce a forecast which is higher than can be explained by previous academic results. We write the forecast as

which is the same as Equation (3), except that the dependent variable is now the forecast F_{j, i} rather than the excess expectation D_{j, i} and also that the female dummy has been deleted.

The reason for not including the female dummy is that we think of the forecast as a combination of two effects: a part based on factual information and a remainder which we identify with overconfidence. This overconfidence will depend on other things, one of which may be gender. We don't observe the remainder (the errors ε_i,i), but we can predict it through the residuals

where α^j,β^j, and γ^j are the least-squares estimates from Equation (5). These residuals thus capture that part of the student's forecast which cannot be explained rationally, and thus correspond to our idea of (over)confidence, which is why we denote them by conf_{j, i}. Note that if we would include the female dummy in Equation (5) then conf and female would be orthogonal to each other, and this is not reasonable.

Overconfidence, thus defined, may include some information which is not available to us, such as private lessons taken before the exam or certain psychological features of the student. Since this information is not available to us we ignore it.

In the first step of the estimation procedure we thus estimate F_{j, i} and obtain the residuals conf_{j, i}. In the second step we regress the exam grades G_{j, i} on the same set of regressors as in Equation (5) and, in addition, on the residuals conf_{j, i} and the female dummy (and a cross term):

The results of the two-step procedure are presented in Table 4. The left panel (columns F₁, F₂, F₃) gives the results of the first step. We see that first-year calculus and (to a lesser extent) statistics are important for F₁, but that home assignments are not important for the forecast. For F₂, the result G₁ of the first exam is important, while first-year calculus remains important as well. For the final exam the forecast F₃ depends much on the results of the earlier two exams G₁ and G₂. The results of first-year calculus (and statistics) are not important anymore; these are absorbed in the grades G₁ and G₂, because recent information is more relevant than older information. We note that Radj2 increases with the exam number, suggesting that with each exam students become more accurate in their forecasts.

The right panel (G₁, G₂, G₃) gives the results of the second step and allows us to test various hypotheses. We see that ϕ is significantly positive at the 1% significance level for all three exams. Its value increases with time/exam number (0.225, 0.282, 0.345), so the impact of overconfidence increases; at least for the men. For the women, the impact decreases (0.396, 0.272, 0.158) when we take the cross term into account. There is evidence in the literature that the more important is the exam the smaller is the overconfidence (Nowell and Alston, 2007). In our case this is true for women but not for men.

The female dummy is not significant and its cross term with conf is significant (at 5%) only for the first exam. In contrast to the results in the left panel (the forecasts), more of the “factual” regressors x_j are significant in the right panel (the grades). The impact of the first-year courses (calculus, statistics, GPA) decreases during the second year, as is to be expected. The grades are significant: G₁ is significant in the G₂ regression (and somewhat less in the G₃ regression), and G₂ is significant in the G₃ regression. Homework results, while not significant for the students' forecasts, are significant for the grades, but only the most recent homework results. The reason, perhaps, is that students understand that homework results are not representative, because there is much collaboration among students and in fact some cheating, so they don't take it into account when forming their forecast. But the plain fact that a student submits the homework (whether own work or not) apparently helps to get a better grade. This finding agrees with Weems (1998), but not with Geide-Stevenson (2009).

Thus we conclude that (a) overconfidence helps in getting a better grade; (b) the impact of information deteriorates quickly over time; (c) homework results are important for the grades, but unimportant for the forecasts; and (d) gender is not significant in exam grades.

We next ask: does overconfidence depend on gender? To answer this question we consider the regression

where we note that conf is orthogonal to the year dummy and the available information in x, because conf is the vector of residuals from Equation (5). Regression results are presented in the Table 5.

Table 5 shows that, given the same objective factors, male students tend to be more optimistic in forecasting their exam grades than female students. The difference ranges from approximately 2.7–5.0 grade points, and this difference seems to decrease over time (within one exam year). Transforming these grade points to a 0–4 scale we divide by 25 and obtain 0.11 and 0.20, which is of the same order as in Ballard and Johnson (2005).

In the previous section we predicted and studied overconfidence as measured by the residuals conf_{j, i} for each student i and exam j. We found that this overconfidence tends to become smaller as the year progresses. We now address this issue in more depth. That is, we ask whether overconfidence decreases, which would mean that students adjust their (over)confidence.

To answer this question we estimate the dynamic regressions

If |θ₂| < 1 then learning takes place between exams 1 and 2. Similarly, if |θ₃| < 1 then learning takes place between exams 2 and 3. We run these regressions separately for men and women, because we have seen that overconfidence is not the same for men and women.

The results are presented in Tables 6, 7. It is clear that adjustment occurs, since |θ_j| is significantly smaller than one. For the adjustment from exam 1 to exam 2 we find θ₂ = 0.19 for women and θ₂ = 0.35 for men. The difference between men and women is statistically significant. For the adjustment from exam 2 to exam 3 we find θ₃ = 0.43 for women and θ₃ = 0.35 for men, and this difference is not statistically significant.

Thus we conclude that (a) confidence adjustment occurs for both male and female students; (b) the adjustment from exam 1 to exam 2 is stronger than the adjustment from exam 2 to exam 3; (c) female students are faster learners, certainly in the step from exam 1 to exam 2; and (d) overconfidence persists (since the values of θ_j are all positive) and this persistency is stronger for men than for women.

We can go one step further. In the above regressions we estimated the average values of the adjustment coefficient for male and female students. But each student is different and the adjustment coefficient may vary from student to student. In order to estimate the individual values of the adjustment coefficient we model θ_j as a function of the individual characteristics of a student:

Inserting Equation (10) in Equation (9) then gives.

Instead of estimates θ^j we now obtain distributions (over i) of estimates.

Kernel density plots for the distribution of the adjustment coefficient are presented in Figure 1 for the adjustment from exam 1 to exam 2) and Figure 2 for the adjustment from exam 2 to exam 3). We see from the first figure that the density plot for women is shifted toward zero, again demonstrating that female students are faster learners (lower persistence) than male students. There is no significant difference between the density plots for the adjustment from exam 2 to exam 3.

The figures provide further (and more detailed) confirmation of our previous conclusions, namely that (a) adjustment takes place; (b) women are faster learners that men; and (c) there is a persistency of overconfidence from one exam to the next, which is stronger for men than for women.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.