Evaluating Efficacy and Motivation of Generative AI Assistants in Learning Functional Dimensioning and Specification of Dimensional Fits

Idoia San Sebastian¹ , Gaizka Erkizia² , Eva Rodriguez¹ , Maider Iturrondobeitia¹ , Pello Jimbert¹

¹Department of Graphic Desing and Engineering Projects, School of Engineering of Bilbao, University of the Basque Country EHU, Spain, ²Department of Graphic Desing and Engineering Projects, School of Engineering of Gipuzkoa, University of the Basque Country EHU, Spain.

How to cite: San-Sebastian, I.; Erkizia, G.; Rodriguez, E.; Iturrondobeitia, M.; Jimbert, P. (2026). Evaluating Efficacy and Motivation of Generative AI Assistants in Learning Functional Dimensioning and Specification of Dimensional Fits. In: 12th International Conference on Higher Education Advances (HEAd’26). Valencia, 15-18 June 2026. https://doi.org/10.4995/HEAd26.2026.21566

Abstract

Functional dimensioning represents a critical competency in first-year engineering education, requiring students to transition from geometric description to functional reasoning. This quasi-experimental study (N=240) evaluated the technical efficacy and motivational impact of Generative Artificial Intelligence in teaching functional dimensioning and ISO 286 fit specification. A comparative design was employed, with an experimental group (n=170) using a structured four-phase prompting framework with a large language model, and a control group (n=70) following traditional instruction. Both groups analysed mechanical assemblies across eight 2-hour sessions. Academic performance was assessed via a blind-graded technical rubric, and motivation through validated questionnaires measuring self-efficacy, intrinsic motivation, perceived competence and engagement. Results indicate that Artificial Intelligence enhances conceptual understanding of functional dimensioning (85% vs. 72% critical dimension identification) while showing mixed calculation accuracy contingent on validation behaviours (systematic validators: 90% vs. minimal validators: 65%). The experimental group demonstrated elevated self-efficacy (d=0.79) though correlation with performance was modest (r=0.30). Findings suggest that properly scaffolded Artificial Intelligence integration reduces affective barriers while requiring explicit instruction in structured prompting and critical validation practices.

Keywords: Artificial Intelligence; Functional Dimensioning; Engineering Education; ISO 286; Computer-Aided Design.

1. Introduction

After the first year of study, engineering students often find it difficult to make the transition from creating geometric models on a computer to documenting what a product should do. They may be adept at creating 3D models. However, they often do not know how to decide on the correct sizes for the parts in a mechanical drawing, which can cause problems when the product needs to be manufactured and assembled. Although special computer programs can provide personalized assistance and alert students when they are making mistakes, little research has been conducted on how to utilize these programs effectively in the first year of engineering study (Baidoo-Anu & Ansah, 2023; Chen et al., 2018).

This study examines how artificial intelligence can be used to teach first-year engineering students about functional dimensioning and adjusting according to the ISO 286 standard (International Organization for Standardization [ISO], 2010). These are skills that students need when working with technical graphics. The study also aims to determine whether AI has an impact on motivation.

Using language models in colleges and universities can greatly benefit students by providing a more personalized learning experience. However, there are also some issues that need to be considered. It is essential to ensure that the information is accurate. This is why it is important to teach students how to use these language models effectively. This means designing the questions asked of the language models carefully so that students can get the most out of them. For example, a step-by-step plan can be used to formulate questions (Mollick & Mollick, 2023). Figure 1 shows the four steps employed in this study to achieve this.

Kasneci et al. (2023) argue that to effectively integrate AI into engineering, students must develop the critical competencies needed to detect the ‘unexpected fragility’ of the system. This theoretical framework underpins the present study’s focus on validation behaviours as a mediating factor in AI-assisted learning.

The present study makes three main contributions. First, it provides empirical evidence on the impact of generative AI on both technical performance and motivation in the specific context of functional dimensioning and ISO 286 fit specification, a domain that has received limited attention in the literature. Second, it identifies validation behaviour as a critical mediating factor in AI-assisted learning, demonstrating that the effectiveness of AI tools depends not merely on their availability but on how students engage with and verify their outputs. Third, it proposes and evaluates a structured four-phase prompting framework designed to scaffold productive AI interaction in technical engineering tasks.

2. Research Questions and Hypotheses

This study addresses four research questions examining technical efficacy and motivational impact of AI assistance in teaching functional dimensioning and ISO 286 fit specification.

RQ1: To what extent does AI assistance improve functional dimensioning reasoning? It is hypothesized that AI enhances identification of critical dimensions and justification quality, though comprehensive specification may remain challenging.

RQ2: To what extent does AI improve ISO 286 fit selection and calculation accuracy? Differential effects are predicted: enhanced performance for complex multi-constraint decisions but conditional calculation accuracy depending on validation behaviors.

RQ3: What effect does artificial intelligence have on how confident students feel in their abilities and how engaged they are in their learning? It is hypothesized that motivation will be increased, but there may be a gap between performance and self-efficacy.

RQ4: What interaction patterns distinguish between successful AI-assisted learning? It is predicted that structured prompting and systematic validation behaviors would predict success.

3. Methodology

This quasi-experimental study employed a non-equivalent control group design evaluating AI-assisted versus traditional instruction across multiple dimensions of learning and motivation.

3.1. Participants and Design

The sample comprised 240 first-year students from the Mechanical Engineering, Electrical Engineering and Industrial Electronics and Automation Engineering programs at the X in the 2024–25 academic year. Table 1 presents the students’ detailed demographic characteristics and group equivalence. There were six instructional groups (Spanish/Basque) and approximately 90 working pairs. Participants were assigned to an Experimental Group (EG, n=170) and a Control Group (CG, n=70). There were eight 2-hour sessions with mechanical assemblies: S1: Introduction/analysis; S2: Sketching; S3: CAD (Solid Edge); S4: Engineering analysis; S5: Technical drawings; S6: Materialization (e.g. 3D printing Artillery Sidewinder H1, PLA, 0.2 mm; CG: Virtual); S7: Technical report; S8: Presentation/evaluation.

Table 1. Demographic Characteristics and Group Equivalence

Characteristic	EG (n=170)	CG (n=70)	Statistical Test
Age (M ± SD)	20.8 ± 2.7	20.6 ± 3.1	t=0.87, p>0.05
Male (%)	85%	86%	χ²=1.23, p>0.05
Mechanical Eng. (%)	45%	43%	χ²=0.35, p>0.05
Electrical Eng. (%)	32%	34%	χ²=0.28, p>0.05
Industrial Electronics (%)	23%	23%	χ²=0.00, p>0.05

3.2. Experimental Conditions

CG: ISO 286 tables (ISO, 2010); textbooks; CAD; office hours. EG: All CG resources, plus a ChatGPT-4 structured four-phase prompting guide (see Figure 1), the required documentation and a 45-minute orientation on LLM capabilities and validation methods. Tasks: (1) Functional dimensioning: identify critical dimensions, create a specification and justify it functionally; (2) ISO 286 fit specification: select fits (H7/g6 and H7/p6), calculate clearances and interferences and justify the selections.

Figure 1. Four-Phase AI Prompting Framework employed in experimental condition.

3.3. Measures

The technical rubric was blind-graded (κ =0.82) and comprised 100 points, with 30 points for functional reasoning, 20 points for specification, 25 points for fit selection and 25 points for calculations. The validated questionnaire was completed after the task and before grading. The following were measured: intrinsic motivation (α=0.84), perceived competence (α=0.79), self-efficacy (α=0.87) and engagement (α=0.76). The EG submitted AI transcripts and validation logs documenting verification actions, detected errors and corrections.

The assessment of motivational impact is grounded in self-determination theory, which distinguishes between the intrinsic and extrinsic dimensions of student engagement. Previously validated measurement instruments were employed in technical education and STEM environments to ensure the reliability of the self-efficacy and perceived engagement scales (López-Fernández et al., 2023; Ryan & Deci, 2020).

4. Results

The results are organized according to the four research questions: examining functional dimensioning accuracy; fit selection and calculations; motivational outcomes; and interaction patterns.

Table 2. Comparison of Technical Performance and Motivational Perception.

Metric	EG	CG	Δ
Critical dimension identification	85%	72%	+13%
Functional justifications	78%	64%	+14%
Dimension omissions	14%	16%	-2%
Simple fit decisions	90%	85%	+5%
Complex multi-constraint decisions	80%	65%	+15%
Self-efficacy (M)	4.0	3.4	d=0.79

4.1. Functional Dimensioning Reasoning (RQ1)

The EG demonstrated a stronger ability to identify functionally critical dimensions (85% vs. 72%) and functional justifications (78% vs. 64%). A qualitative review revealed that EG justifications more frequently referenced specific functional consequences (e.g. H7/G6 ensures clearance, enabling rotation), rather than generic statements. However, the proportion of dimension omissions was similar (14% vs. 16%), indicating that achieving comprehensive specifications remains challenging regardless of support.

4.2. ISO 286 Fit Selection and Calculations (RQ2)

Similar performance was shown for straightforward decisions (EG: 90%, CG: 85%). However, complex, multi-constraint decisions favoured the EG (80% versus 65%), suggesting that AI is particularly beneficial for conceptually demanding coordination. Calculation accuracy revealed a critical dependency on validation: systematic validators achieved 90% accuracy, compared to 65% for minimal validators — a 25-percent-point difference (see Figure 2).

Figure 2. Calculation accuracy by validation profile.

4.3. Motivation and Self-Efficacy (RQ3)

The EG reported higher levels of self-efficacy (M = 4.0 vs. 3.4; d = 0.79; large effect size), enjoyment (M = 3.7 vs. 3.2), and competence (M = 3.8 vs. 3.3). Qualitative analysis revealed that the EG valued immediate explanations, judgement-free interaction and breaking overwhelming problems down into manageable steps. The CG expressed uncertainty about correctness, confusion regarding ISO tables and time pressure. However, the correlation between self-efficacy and performance was modest (r = 0.30, p < 0.05), accounting for only 9% of the variance. This suggests that AI enhances confidence without proportional skill gains, raising concerns about overconfidence.

4.4. Interaction Patterns and Validation (RQ4)

Structured iterative prompting achieved a task quality of 85%, compared to 70% for single-shot approaches. Validation profiles revealed distinct performance patterns, as shown in Table 3. The 20-point difference between the performance of systematic and minimal validators indicates that the effectiveness of AI critically depends on metacognitive validation practices. High performers documented multi-step verification, whereas low performers accepted AI outputs without validation.

Table 3. Validation Profiles and Performance Outcomes

Profile	n	Verification Rate	M Score	SD
Systematic Validators	60	≥75% of outputs	88	6.2
Selective Validators	70	40-74% of outputs	78	8.5
Minimal Validators	40	<40% of outputs	68	11.3

5. Discussion

The results of this study support Baidoo-Anu and Ansah’s (2023) warning not to confuse AI fluency with technical precision. This explains the ‘illusory competence’ observed in students who displayed high confidence yet produced inaccurate unvalidated calculations.

AI can significantly enhance functional reasoning by serving as an accessible ‘more knowledgeable other’ that can bridge the gap between geometric and functional reasoning. The superior EG performance in identifying critical dimensions and providing conceptual justifications (see Table 2) suggests that AI excels in explanatory and reasoning-scaffolding aspects. However, a lack of mathematical precision creates vulnerability for first-year students who lack expertise in error detection. The 25-point validation gap (Figure 2, Table 3) reveals a pedagogical paradox: those who need AI the most are the least able to use it critically due to their lack of domain knowledge. The substantial motivational benefits (d = 0.79) reduce affective barriers by providing immediate, judgement-free feedback and addressing psychological safety in learning environments.

However, the modest self-efficacy–performance correlation (r = 0.30) raises concerns about illusory competence — students may develop inflated confidence without developing the corresponding skills. This highlights the need for explicit calibration exercises in which students compare AI-assisted solutions with verified expert solutions, analyze discrepancies and reflect on instances where AI was helpful versus misleading. While the structured, four-phase prompting guide (Figure 1) appears effective, it requires active engagement; mere tool availability is insufficient. Effective integration requires the teaching of AI literacy as an explicit technical communication skill, including the formulation of contextually rich queries, the request for reasoning rather than answers and the systematic validation against authoritative sources.

The observed improvement in conceptual reasoning versus vulnerability in calculation precision can be explained by cognitive load theory. AI reduces extraneous cognitive load by facilitating the search for information, but students must possess solid mental schemas for validation. As Kasneci et al. (2023) point out, LLMs have an inherent ‘brittleness’ which means students need to develop AI literacy with a focus on critical thinking and constantly verifying results (Sweller et al., 2019).

These findings are also consistent with previous research cautioning that, although AI can transform access to information, its effectiveness in technical disciplines hinges on users’ capacity to identify factual inaccuracies (Adiguzel et al., 2023). This may explain why the experimental group demonstrated high self-efficacy (M = 4.0), which did not always translate into technical precision in the absence of systematic validation (see Table 3).

6. Conclusions

When adequately supported by pedagogical resources, artificial intelligence assistants can effectively help first-year engineering students by improving their conceptual understanding of complex problems, validating their calculations and providing significant motivational benefits (d = 0.79). However, explicit instruction on verification protocols must be provided.

Practical recommendations include: (1) integrating AI with clear guidelines that differentiate between uses such as explaining concepts and supporting reasoning, which require less rigorous validation, and uses such as calculations and regulatory compliance, which require more rigorous validation; (2) explicitly teaching the four-phase prompting method (Figure 1), which involves contextualizing queries, requesting explanations before solutions, checking against authoritative sources and systematically validating results; (3) designing assessments that require validation documentation, value the identification of AI errors positively and include critical reflections on usefulness and limitations; and (4) providing calibration exercises with deliberate errors to develop metacognitive awareness.

First-year students require more support than advanced students because they have limited knowledge of the subject area. While the integration of AI offers opportunities to democratize personalized teaching and reduce affective barriers, it also poses challenges related to illusory competence, dependence on external tools and undetected errors. To succeed, deliberate pedagogical design is required, along with appropriate support structures, explicit validation protocols and structured instruction. Passive technological adoption must be avoided.

References

Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), ep429. https://doi.org/10.30935/cedtech/13152

Baidoo-Anu, D., & Ansah, L. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52–62. https://doi.org/10.61969/jai.1337500

Chen, X., Breslow, L., & DeBoer, J. (2018). Analyzing productive learning behaviors for students using immediate corrective feedback in a blended learning environment. Computers & Education, 117, 59–74. https://doi.org/10.1016/j.compedu.2017.09.013

International Organization for Standardization. (2010). ISO 286-1:2010 Geometrical product specifications (GPS)—ISO code system for tolerances on linear sizes—Part 1: Basis of tolerances, deviations and fits.

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., . . . Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274. https://doi.org/10.1016/j.lindif.2023.102274

López-Fernández, D., Gordillo, A., Pérez, J., & Tovar, E. (2023). Learning and motivational impact of game-based learning: Comparing face-to-face and online formats on computer science education. IEEE Transactions on Education, 66(4), 360–368. https://doi.org/10.1109/TE.2023.3241099

Mollick, E. R., & Mollick, L. (2023). Assigning AI: Seven approaches for students, with prompts (The Wharton School Research Paper, Pedagogy Collection). https://doi.org/10.2139/ssrn.4475995

Ryan, R. M., & Deci, E. L. (2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, Article 101860. https://doi.org/10.1016/j.cedpsych.2020.101860

Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5