Publication
Investigating Large Language Models ability to evaluate heading-related accessibility barriers
| datacite.subject.fos | Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática | pt_PT |
| dc.contributor.advisor | Duarte, Carlos Alberto Pacheco dos Anjos | |
| dc.contributor.advisor | Pereira, Letícia Seixas | |
| dc.contributor.author | Costa, Miguel Santos Pedro Soares da | |
| dc.date.accessioned | 2025-01-10T15:55:57Z | |
| dc.date.available | 2025-01-10T15:55:57Z | |
| dc.date.issued | 2024 | |
| dc.date.submitted | 2024 | |
| dc.description | Tese de Mestrado, Engenharia Informática, 2024, Universidade de Lisboa, Faculdade de Ciências | pt_PT |
| dc.description.abstract | This thesis investigates the ability of Large Language Models (LLMs) to identify and evaluate heading-related accessibility barriers in HTML documents, with a focus on improving web accessibility. Headings play a crucial role in ensuring web content is accessible, yet they are often misused, leading to barriers that impact users, particularly those relying on assistive technologies. The study aims to evaluate how effectively LLMs, such as GPT-4o and Llama 3.1, can detect specific heading-related issues. To achieve this, I identified and categorized common heading-related accessibility issues, such as ”Excessive Use of Headings”, ”Misleading or Confusing Headings”, and ”Empty Accessible Name”. A set of test webpages was created to simulate these issues, and specific questions were designed to prompt the LLMs to analyze and evaluate these accessibility barriers. The thesis explores the accuracy and consistency of LLMs in responding to these prompts, highlighting their ability to address certain issues while encountering limitations in others. The study further examines the impact of context, session, and queue structures—Singular and Connected Queues—on LLM performance. Results show that including contextual information significantly improves LLM accuracy, while session data and queue structures have a lesser impact. Overall, the thesis presents a system for improving LLMs’ ability to evaluate webpages’ accessibility and suggests future work on refining prompts and leveraging LLMs for real-time accessibility repairs. This research contributes to the ongoing exploration of how LLMs can enhance web accessibility, specifically in addressing heading-related issues, offering a foundation for further advancements in LLM-driven evaluation and remediation systems. | pt_PT |
| dc.identifier.tid | 203875362 | |
| dc.identifier.uri | http://hdl.handle.net/10400.5/97081 | |
| dc.language.iso | eng | pt_PT |
| dc.subject | Acessibilidade na Web | pt_PT |
| dc.subject | Grandes Modelos de Linguagem | pt_PT |
| dc.subject | Títulos | pt_PT |
| dc.subject | Chat-GPT | pt_PT |
| dc.subject | Llama | pt_PT |
| dc.subject | Teses de mestrado - 2024 | pt_PT |
| dc.title | Investigating Large Language Models ability to evaluate heading-related accessibility barriers | pt_PT |
| dc.type | master thesis | |
| dspace.entity.type | Publication | |
| rcaap.rights | openAccess | pt_PT |
| rcaap.type | masterThesis | pt_PT |
| thesis.degree.name | Mestrado em Engenharia Informática | pt_PT |