Measuring Digital Self-Efficacy in International Large-Scale Assessments: An International Comparison Between ICILS and PISA

Daniel Miranda, Ismael Aguayo, Juan Carlos Castillo, Nicolas Tobar y Tomás Urzúa

University of Chile y Millennium Nucleus on Digital Inequalities and Opportunities.

X Seminar “Data from and for the educational system: tools for research and teaching”, Ostia, Rome, 19 - 20 - 21 November 2025

NUDOS

More information: nudos.cl

Starting point

Digital self-efficacy and gender

“… expectations about one’s capabilities to learn and accomplish tasks in digital technologies and digital environments, is one of the principal components to promote the formation of digital competences” (Ulffert-Blank & Schmidt, 2022).

Bidimensional digital self-efficacy?

  • A changing concept: Computacional -> Internet -> TICs -> Digital (DigComp)

  • In the last years it has been proposed a distinction between two types of self-efficacy: general and specialized.

  • Cross-country studies operationalize digital self-efficacy in a one-dimensional and two-dimensional manner

Country differences?

It is possible to identify two dimensions on PISA Digital Self-efficacy measurement?

Is the bidimensional model of Digital Self-efficacy equivalent by gender and across countries?

Which gender differences exist on Digital Self-efficacy across countries?

Data

  • Programme for International Student Assessment 2022 (n≈183.000).

  • International Computer and Information Literacy Study 2023 (n≈90.000).

  • 22 shared countries.

Baterías de Autoeficacia digital

PISA: ¿To what extent are you able to do the following tasks when using ?

ICILS: How well you can do…?

General Self_Efficacy

PISA: 8 ítems, ICILS: 10 ítems

  • Search for and evaluate information online.

  • Share content online.

  • Write or edit text.

  • Create or edit images.

Specialized Self-Efficacy

PISA: 6 ítems, ICILS: 3 ítems

  • Website development.

  • Programming.

  • Computational reasoning.

Methods

  • Conformatory Factor Analizes and re-specification

  • Stability among countries and gender (Multigroup CFA)

  • Country and gender differences

Results

  1. Measurement model
  2. Invariance testing
  3. Mean distributions across countries
  4. Gender differences across countries

Original model: pooled

PISA 2022

CFI = 0.98; TLI = 0.97; RMSEA = 0.15, exceeding the maximum acceptable RMSEA threshold of 0.08

ICILS 2023

CFI = 0.97; TLI = 0.96; RMSEA = 0.10, exceeding the maximum acceptable RMSEA threshold of 0.08

Adjusted Modelo: pooled

AFC and Invariance

Across countries

By gender

Conclusion

  • Is it possible to identify two dimensions in PISA and ICILS?

    • Yes, but with fewer items than those proposed by both studies.

  • Is the two-dimensional model of digital self-efficacy equivalent across genders and countries?

    • In PISA, yes; in ICILS, only at the metric level.

  • What gender differences exist in digital self-efficacy across countries?

    • Women have an advantage in general self-efficacy, and men in specialized self-efficacy.

Discussion

  • The evolution of measurements: ICTSE vs. DSE and the DigComp framework

  • Are there two or more dimensions?

  • The counterintuitive relationship between countries’ development and the gender gap

Next steps

  • What macro-level factors can explain the differences in digital self-efficacy and the gender gap?

  • What is happening in developed/developing countries?

Thank you!

Anex

Eliminated items

  • Multimedia manipulation
  • Searching for information sources
  • Privacy settings
  • App selection.