Logo do repositório
 
Publicação

Technical note: “Bit by bit”: a practical and general approach for evaluating model computational complexity vs. model performance

2021-03Artigo científico Acesso aberto
Ver registo simples
dc.contributor.authorAzmi, Elnaz
dc.contributor.authorEhret, Uwe
dc.contributor.authorWeijs, Steven V.
dc.contributor.authorRuddell, Benjamin L.
dc.contributor.authorPerdigão, Rui A. P.
dc.date.accessioned2022-01-29T17:12:10Z
dc.date.available2022-01-29T17:12:10Z
dc.date.issued2021-03
dc.description.abstractOne of the main objectives of the scientific enterprise is the development of well-performing yet parsimonious models for all natural phenomena and systems. In the 21st century, scientists usually represent their models, hypotheses, and experimental observations using digital computers. Measuring performance and parsimony of computer models is therefore a key theoretical and practical challenge for 21st century science. “Performance” here refers to a model’s ability to reduce predictive uncertainty about an object of interest. “Parsimony” (or complexity) comprises two aspects: descriptive complexity – the size of the model itself which can be measured by the disk space it occupies – and computational complexity – the model’s effort to provide output. Descriptive complexity is related to inference quality and generality; computational complexity is often a practical and economic concern for limited computing resources. In this context, this paper has two distinct but related goals. The first is to propose a practical method of measuring computational complexity by utility software “Strace”, which counts the total number of memory visits while running a model on a computer. The second goal is to propose the “bit by bit” method, which combines measuring computational complexity by “Strace” and measuring model performance by information loss relative to observations, both in bit. For demonstration, we apply the “bit by bit” method to watershed models representing a wide diversity of modelling strategies (artificial neural network, auto-regressive, processbased, and others). We demonstrate that computational complexity as measured by “Strace” is sensitive to all aspects of a model, such as the size of the model itself, the input data it reads, its numerical scheme, and time stepping. We further demonstrate that for each model, the bit counts for computational complexity exceed those for performance by several orders of magnitude and that the differences among the models for both computational complexity and performance can be explained by their setup and are in accordance with expectations. We conclude that measuring computational complexity by “Strace” is practical, and it is also general in the sense that it can be applied to any model that can be run on a digital computer. We further conclude that the “bit by bit” approach is general in the sense that it measures two key aspects of a model in the single unit of bit. We suggest that it can be enhanced by additionally measuring a model’s descriptive complexity – also in bit.pt_PT
dc.description.versioninfo:eu-repo/semantics/publishedVersionpt_PT
dc.identifier.citationAzmi, E., Ehret, U., Weijs, S. V., Ruddell, B. L., and Perdigão, R. A. P.: Technical note: “Bit by bit”: a practical and general approach for evaluating model computational complexity vs. model performance, Hydrol. Earth Syst. Sci., 25, 1103–1115, https://doi.org/10.5194/hess-25-1103-2021, 2021.pt_PT
dc.identifier.doi10.5194/hess-25-1103-2021pt_PT
dc.identifier.issn1607-7938
dc.identifier.urihttp://hdl.handle.net/10451/51035
dc.language.isoengpt_PT
dc.peerreviewedyespt_PT
dc.publisherCopernicus Publicationspt_PT
dc.relationHelmholtz Associationpt_PT
dc.relationDeutsche Forschungsgemeinschaftpt_PT
dc.relationKarlsruhe Institute of Technologypt_PT
dc.relationFCT UIDB/00329/2020pt_PT
dc.relationFCT UID/EEA/50008/2019pt_PT
dc.relationNorthern Arizona Universitypt_PT
dc.relation.publisherversionhttps://hess.copernicus.org/articles/25/1103/2021/pt_PT
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/pt_PT
dc.titleTechnical note: “Bit by bit”: a practical and general approach for evaluating model computational complexity vs. model performancept_PT
dc.typejournal article
dspace.entity.typePublication
oaire.citation.endPage1115pt_PT
oaire.citation.issue2pt_PT
oaire.citation.startPage1103pt_PT
oaire.citation.titleHydrology and Earth System Sciencespt_PT
oaire.citation.volume25pt_PT
person.familyNamePerdigão
person.givenNameRui A. P.
person.identifier.ciencia-id9817-6C5B-1956
person.identifier.orcid0000-0001-5543-1754
person.identifier.ridK-9896-2015
person.identifier.scopus-author-id16025322100
rcaap.rightsopenAccesspt_PT
rcaap.typearticlept_PT
relation.isAuthorOfPublication1d0aaa34-2a0b-4161-95a2-ac966a4a3b68
relation.isAuthorOfPublication.latestForDiscovery1d0aaa34-2a0b-4161-95a2-ac966a4a3b68

Ficheiros

Principais
A mostrar 1 - 1 de 1
Miniatura
Nome:
hess-25-1103-2021.pdf
Tamanho:
1.63 MB
Formato:
Adobe Portable Document Format
Ver/Abrir

Coleções

cE3c - Artigos em Revistas Internacionais