# Dr. Benedikt Holm

> Icelandic AI engineer and researcher. PhD in Computer Science from Reykjavík University. Specialises in signal processing, clinical AI, data infrastructure, and real-time platforms. Based in Germany.


Query: "your question"


## Result 1 — scorecraft (score: 0.197)

Nath, et al. 2024. “Temporal and Sleep Stage- Dependent Agreement in Manual Scoring of Respiratory Events.” Journal of Sleep Research 34: e14391. R Core Team. 2023. “R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.” https:// www. R- proje ct. org/ . Journal of Sleep Research, 2026 13 of 13 Rayan, A., A. B. Szabo, and L. Genzel. 2023. “The Pros and Cons of Using Automated Sleep Scoring in Sleep Research: Comparative Analysis of Automated Sleep Scoring in Human and Rodents: Advantages and Limitations.” Sleep 47, no. 1: zsad275. https:// doi. org/ 10. 1093/ sleep/ zsad275. Rosenberg, R. S., and S. Van Hout. 2014. “The American Academy of Sleep Medicine Inter- Scorer Reliability Program: Respiratory Events.” Journal of Clinical Sleep Medicine 10, no. 4: 447–454. Rusanen, M., H. Korkalainen, H. Gretarsdottir, et al. 2024. “Self- Applied Somnography: Technical Feasibility of Electroencephalography and Electro- Oculography Signal Characteristics in Sleep Staging of Suspected Sleep- Disordered Adults.” Journal of Sleep Research 33, no. 2: e13977. Rusanen, M., H. Korkalainen, H. Grétarsdóttir, et al. 2023. “Self- Applied Somnography: Technical Feasibility of Electroencephalography and Electro- Oculography Signal Characteristics in Sleep Staging of Suspected Sleep- Disordered Adults.” Journal of Sleep Research 33: e13977. https:// doi. org/ 10. 1111/ jsr. 13977 . Sprague, R. H. 1980. “A Framework for the Development of Decision Support Systems.” MIS Quarterly 4, no. 4: 1–26. http:// www. jstor. org/ stable/ 248957. Varsha, P. S. 2023. “How Can We Manage Biases in Artificial Intelligence Systems—A Systematic Literature Review.” International Journal of Information Management Data Insights 3, no. 1: 100165. https:// doi. org/ 10. 1016/j. jjimei. 2023. 100165. Virtanen, P., R. Gommers, T. E. Oliphant, et al. 2020. “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python.” Nature Methods 17, no. 3: 261–272. https:// doi. org/ 10. 1038/ s4159 2- 019- 0686- 2. Wara, T. U., A. H. Fahad, A. S. Das, and M. M. H. Shawon. 2024. “A Systematic Review on Sleep Stage Classification and Sleep Disorder Detection Using Artificial Intelligence.” arXiv [cs.LG]. Zeng, L. N., Q. Q. Zong, Y. Yang, et al. 2020. “Gender Difference in the Prevalence of Insomnia: A Meta- Analysis of Observational Studies.” Frontiers in Psychiatry 11: 577429. https:// doi. org/ 10. 3389/ fpsyt. 2020. 577429. Appendix A Study Completion Questionnaire Participant Information 1. What is your name? (We will not use your personal identity for any analysis or publication, and it will remain completely confidential.) Answer: _______________________________. 2. What is your email? (If this is not prefilled, please use the same email you used to log into MicroNy x .) Answer: _______________________________. 3. Where are you working? (Center, City, Country) Answer: _______________________________. Study Awareness 1. I heard about this study from: • The invitation email sent to the Sleep Revolution. • The talk held by Dr. Erna Sif Arnardóttir at the Sleep Europe conference. • The talk held by Dr. Erna Sif Arnardóttir at the ERS conference. • Email sent to the members of the ESST. • Other: _____________________. 2. Are you part of the Sleep Revolution? • Ye s • No User Experience This section of the questionnaire is designed to gain insight into the ease of use of the MicroNyx platform. 1. The system was easy to use. Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree. 2. I felt that I could reliably read, interpret and score the signals in the interface. Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree. 3. The scoring recommendations were easy to see. Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree. 4. It was clear which scoring recommendations were from a human and which were from an AI. Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree. 5. I felt the recommendations were helpful when scoring. Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree. Additional Feedback If you have any further comments, please write them here. We are very happy to hear your feedback. Answer: _______________________________.

## Result 2 — pipeline (score: 0.167)

three distinct components: 1. Web-based front-end for user uploads, administration, and dispatching of jobs to the other components (henceforth referred to as thefront end). 2. Three-night PSG splitter (henceforth referred to as thesplitter). 3. Processing pipeline that augments PSGs with automatic scorings (henceforth referred to as theprocessor). FrontiersinNeuroinformatics03frontiersin.org Holm et al.10.3389/fninf.2024.1379932 TABLE 1Comparison of contributions of this work and similar work. WorkAddresses integration into existing work environments Addresses impact on workflow Addresses opinions of medical professionals on AI in workflow Our workXXX Rayan et al.(2023)X Choo et al.(2023)X Oxholm et al.(2021)X Gerla et al.(2018)X Gerla et al.(2019)X Figure 1shows an overview of the platform architecture. An important feature of the platform is to allow users (e.g., sleep technologists and healthcare professionals) to upload multiple PSGs to be shared and scored at the same time without breaking the platform. To achieve this, the FastAPI Python web framework was used, which despite its simplicity handles multi-user web applications supporting asynchronous code ( tiangolo, 2023). The platform is protected with a user login access in which each user is a validated member of Sleep Revolution consortium ( Sle, 2021). Additionally, the front end handles receiving signals from both the splitter and the processor via HTTP requests and issuing jobs to the splitter when a new PSG is received and to the processor when a PSG has successfully been split. Splitting is necessary when several nights’ PSG are combined into one file. The job queue was achieved using a RabbitMQ queuing server, which is a program that allows disparate asynchronous programs to communicate by listening and issuing messages to a queue ( RabbitMQ Contributors, 2007). By utilizing a message-queue protocol, the font end can offload more time and memory-consuming projects such as generating automatic scorings to other processes, thus reducing the probability of users experiencing downtime, or data loss. The processor is the final component of the architecture. Its purpose is to prepare the individual night PSG by augmenting the PSG with the AI scoring, along with the gray area scoring. The output of the processor is twofold. Firstly, the processor prepares a “scoring" version of the PSG that is augmented with predicted sleep stages from an automatic scoring algorithm integrating gray areas and is made available for manual scoring, and a version meant for later computer processing and machine learning. Each component was containerized using the virtualization software Docker ( Merkel, 2014) for enhanced isolation, consistency, and reproducibility during deployments, which is important in sustainable and secure development. As introduced previously, the processor prepares the PSG to be manually scored, stored, and ready for further analysis. To reduce the manual work of the sleep technologists, a crucial step in the processor is highlighting areas in the PSG that are hard to score for the algorithm, i.e. gray areas. The gray area augmentation works first by sending each one-night PSG EDF file to the trained deep learning model aSAGA ( Rusanen et al., 2023). The aSAGA architecture is based on a revisited U-time architecture for scoring and respiratory events prediction ( Perslev et al., 2021;Huttunen et al., 2022 ). The U-time is an encoder-decoder structure consisting of blocks of consecutive convolutional, batch normalization, and pooling layers. However, in the aSAGA algorithm, a single-channel model is used, which was first trained on PSGs’ EEG (C4-M1) and then fine-tuned with an EOG (E1-M2) channel using self-applied PSGs with frontal setup. This was done to have generalizability between EEG and EOG channels and to increase the compliance of the model for frontal EEG and EOG setups. The aSAGA model is parameterized to return a hypnogram of the same length as the number of epochs from the signal input. The model has an accuracy of 80% estimated over different scored sleep datasets. This accuracy is on par with manual scoring Nikkonen et al.(2024), however, the gray areas from aSAGA model prediction have been validated by comparing the match with the gray areas from predicted manual scoring uncertainty. The second part concerns the gray areas. Using the predicted hypnodensity from the aSAGA model as input, a trained clustering algorithm tags each epoch that belongs to the gray areas ( Jouan et al., 2023 ). The clustering algorithm is a multi-objective method based on multinomial mixture models clustering the different levels of sleep technologist agreement and summarizing the results into two sets of high-agreement andgray areaclusters. The threshold is selected according to the maximization of the distance between two distributions of the sleep technologist’s agreement measure. When the algorithm receives a new hypnodensity, it outputs a hypnogram called aSAGA-UA withgray area. Figure 2Billustrate such predicted hypnogram where eachgray areaare represented as a line discontinuity named “whitespace.” Using aSAGA-UA, it becomes easier for the sleep technologists to view epochs where the AI scoring may not be accurate, and need to be re-evaluated. In Figure 2Abetween 12:30 a.m. and 1:30 there are many transitions between Wake, N1, N2, and REM scored by the algorithm. For the same time period in Figure 2B, there are many line discontinuities characterized by a whitespace symbolizing gray areas. For instance, the predicted N1 and REM sleep stages in Figure 2Aare not present inFigure 2B where whitespaces are clearly visible instead. Regarding the high number of sleep transitions happening in a few minutes, the associated signal might be hard to interpret by the algorithm. A manual review from the sleep technologist is needed in that part of the hypnogram. The method has been evaluated on a real case of uncertainty analysis of 50 PSGs manually scored by 10 sleep technologists. We refer to this dataset as 50×10PSG. This dataset comes from a cohort of 50 participants that have previously been scored by ten independent sleep technologists to create a consensus scoring ( Jouan et al., 2023;Rusanen et al., 2023 ). After testing the clustering algorithm on predicted hypnodensities from aSAGA, the threshold separating the gray FrontiersinNeuroinformatics04frontiersin.org

## Result 3 — msc (score: 0.163)

rate of breathing, and is therefore an estimation of the length of a possible target breath cycle within the w[n]. To ensure that its estimation was as accurate as possible, it was vital to prevent the autocorrelation from creating artificial peaks due to the presence of artifacts in 8CHAPTER 2. BREATH SYNCHRONOUS SEGMENTATION the signal. To achieve this, the RIP signal within the window was smoothed using a Savitzky-Golay filter, which fitted a polynomial function to smooth the data points [47]. The filter employed used a third order polynomial and a window size of 51. The filter parameters were tuned beforehand via experimentation to ensure that the smoothing had a minimal effect on the overall shape of the RIP signal while still eliminating some of the finer noise. Additionally, each individual RIP signal window was corrected for skew. This was achieved by fitting a linear function to the signal, then adjusting the function such that its y-intercept was 0.0. The function was then subtracted from each sample of the signal. This procedure removes large-scale skew from the signal window, but leaves the general shape of the RIP signal intact. The procedure also had a positive effect on the template waveform fitting procedure, making it less likely to produce incorrect results due to skew. The result of this pre-processing step was that the cleaned thoracic RIP signal was ready to be used to estimate the breath lengths. Figure 2.2 shows a thorax signal before and after smoothing and skew removal. Figure 2.2: Thoracic RIP before and after smoothing and skew removal. 2.3 Main Algorithm Body In the first step, the algorithm took an analysis windoww[n], containing a cleaned signal and estimated its periodicityTusing the autocorrelation function, which is commonly used to estimate a signal’s periodicity. The principle of the autocorrelation function (ACF) is to shift the signal forwards in time bykand to compare it to itself. Whenk= 0, the signal correlates perfectly with itself, but askincreases, the correlation decreases. The formula for autocorrelation of a signalxis: ACF(x)[k] = 1 N N−1 ∑ n=0 x[k]·x[n+k],(2.1)

## Result 4 — pipeline (score: 0.162)

conclude that while the inclusion of AI into the workflow of sleep technologists can have a positive impact in terms of speed and agreement, there is a need for trust in the algorithms. KEYWORDS sleep research, platform, human-in-the-loop, machine learning, scoring time, agreement, explainable AI, trust Frontiersin Neuroinformatics01frontiersin.org Holm et al.10.3389/fninf.2024.1379932 Introduction The emergence of explainable artificial intelligence (XAI) presents vast potential for revolutionizing various application areas, such as in healthcare ( De Bock et al., 2023). However, despite the great potential, there are significant issues that need to be tackled before XAI can be fully utilized ( Jermutus et al., 2022). One such issue originates from application areas within healthcare, where automation of manual tasks and data-driven decision-support has to take the central stage before XAI can become a viable option ( Loh et al., 2022 ). A subfield of healthcare is the collection and analysis of sleep recordings, referred to as polysomnography (PSG) ( Arnardottir et al., 2021 ). A PSG is an overnight recording of various biomedical signals, such as an electroencephalogram (EEG), electromyogram (EMG), electrooculogram (EOG), and various respiratory signals. Upon collection, the PSG must be manually annotated by a sleep technologist which is a cumbersome and time- consuming task ( Arnardottir et al., 2021). PSG scoring is a vital step in the process of identifying and diagnosing the presence of many sleep disorders, some of which are extremely prevalent ( Benjafield et al., 2019 ). A sleep technologist will manually review the recording according to a set of rules devised by the American Academy of Sleep Medicine (AASM), labeling events such as respiratory events, and sleep stages in a process referred to as scoring. The sleep stage scoring is done by assigning a specific class to each 30-s segment (also called epochs) in the recording. The sleep stages classes are categorized into five categorical values: the Wake (W) class for wake period, the rapid eye movement category (REM) and three non- REM stages (N1, N2, and N3) that respectively describe the depth of sleep. A product of the PSG scoring is the creation of a hypnogram, a graphical representation tracing the progression of sleep stages throughout the night. This visual tool, often complemented by a hypnodensity graph, provides a detailed overview of the patient’s sleep architecture, capturing transitions between sleep stages ( Jang et al., 2022 ;Pevernagie and Arnardottir, 2024). Self-applied-PSG (henceforth referred to as simply PSG), a newly designed simplified version of traditional PSG, utilizing frontal EEG instead of the conventional International 10–20 System, refers to a type of sleep study that the participant can set up themselves and sleep with at home for up to three nights in the current work ( Arnardottir et al., 2022 ). One of the main drawbacks of the current scoring process is, as stated earlier, that it can be excessively time-consuming, which can cause considerable delays in providing sleep reports to healthcare providers and consequently delay diagnosis ( Biedebach et al., 2024), as well as increase the cost of healthcare considerably ( Wickwire, 2021 ). Adding to this challenge, significant inter-scorer variability exists ( Nikkonen et al., 2024); disagreements can reach 19.3% for sleep stages ( Nikkonen et al., 2024) and 11.6% for respiratory events ( Redline et al., 2007). Delays and disagreements such as these can have negative effects on patient outcomes, as untreated sleep disorders can have a significantly negative impact on patient health ( Dikeos and Georgantopoulos, 2011). The advent of machine learning and other automatic scoring algorithms offers a potential solution by automating the process of manual scoring, which the AASM sees great potential in Goldstein et al.(2020). However, the development and application of machine learning are often prohibitively technical, requiring diverse knowledge of computer science to achieve ( Giray, 2021; Brennan and Kirby, 2022). There is also a dire need for socio-technical alignment, i.e. the multi-disciplinary collaboration between the computer scientists integrating the algorithms, and the professionals working in the context in which the algorithms are being integrated ( Brennan and Kirby, 2022). The integration of AI, machine learning, or advanced data-driven decision-making of any kind into the workflow may move the industry professionals from a generative role (creating the outputs themselves) to the role of auditors, where they correct the output of the algorithms, and consult with computer scientists to tweak and alter the models to handle edge cases or incorrect generations by the algorithm ( Grønsund and Aanestad, 2020). Moreover, in the rare case when socio-technical alignment is reached, trust issues often surface, where the professionals working within the context that the algorithms are integrated into, may not trust the outcomes ( Islind and Hult, 2022 ), which has posed a great limitation in healthcare ( Lee and Yoon, 2021;Jermutus et al., 2022). This mistrust has received limited focus in terms of research contributions and needs to be studied further. Machine learning models are often deemed a “black box,” owing to their lack of transparency and the extensive technical knowledge needed to understand them. Moreover, their incapacity to adapt to dynamically evolving requisites often leads to their obsolescence. This has resulted in the increasing prevalence of human-in-the- loop AI systems ( Mosqueira-Rey et al., 2023). Human-in-the-loop AI systems allow one or more human experts to take an active part in the training process by continuously evaluating the model and providing new inputs that are then selectively used to re-train the model in a process called active learning ( Settles, 2009). To advance and modernize sleep research as well as to enable the collection of a large-scale European sleep recording dataset, the Sleep Revolution project, a joint venture involving 24 European partners, was initiated ( Arnardottir et al., 2022). Each partner contributes∼60 sets of three-night PSGs. Sleep technologists then evaluate these on a shared workstation which is a part of the Sleep Revolution high-performance cluster. After this, healthcare professionals analyze sleep parameters, which helps them to diagnose the patient. A significant objective of the Sleep Revolution is to reduce scoring time ( Arnardottir et al., 2022). One strategy to achieve that goal is to direct the focus of the sleep technologists to the areas of sleep that automatic algorithms have less “certainty” of. By displaying these areas of high uncertainty, referred to asgray areasfrom now on, we can specifically target the sleep technologists toward these areas, instead of unilaterally trusting or mistrusting the automatic scoring algorithms ( Jouan et al., 2023). Most of the research done on automatic sleep staging algorithms mainly focuses on the increase in model prediction accuracy or agreement. With recent datasets mobilizing an ensemble of independent sleep technologists scoring the same record, research on uncertainty quantification, such as gray area identification in the domain of sleep staging, is growing ( Bakker et al., 2023 ;Rusanen et al., 2023;Nikkonen et al., 2024). However, the union of sleep staging algorithms, including selectively focusing the attention of sleep technologists using uncertainty or gray areas during sleep scoring, is a newborn concept that needs to be assessed. FrontiersinNeuroinformatics02frontiersin.org

## Result 5 — scorecraft (score: 0.161)

Participants The study was heavily dependent on the participation of sleep technologists. To recruit sleep technologists as participants in the scoring collection, an email invitation was sent to 49 sleep technologists affiliated with the Sleep Revolution (Arnardottir FIGURE 3 | The MicroNyx scoring interface displaying a traditional PSG. Journal of Sleep Research, 2026 5 of 13 et al. 2022) project. In addition, the study was advertised at con- ferences in the European Respiratory Society (ERS) and the European Sleep Research Society (ESRS) in September 2024. An invitation was also emailed to European Society of Sleep Technologists members via a newsletter. In total, 16 sleep tech- nologists across Europe participated in the study. 2.6 | Study Procedure Each technologist was instructed to complete a 10- min (20- epoch) tutorial session in which they were shown how to navigate the scoring interface, how recommendations appeared, and how to score using the MicroNyx interface successfully. After the tutorial, each technologist was directed to complete two scoring sessions, one for PSG and one for self- applied PSG, where they scored ex- actly 1 h of sleep data per session. After those two sessions, the sleep technologists were instructed to wait a week before scoring two additional sessions for the same recordings. The waiting time was instructed to prevent familiarity with the recordings. The same 1- h segment was used for both the traditional and self- applied scoring sessions, as the two PSG types were re- corded in parallel on the same individual. This ensured that participants scored simultaneous data captured by different equipment, allowing for direct comparison between tradi- tional and self- applied PSG without variability in underlying sleep physiology. After successfully completing the four sessions, a link to a short post- study survey was presented to the sleep technologists. The post- study survey aimed to gauge how each sleep technologist perceived the MicroNyx platform for its ease of use and ability to score sleep stages on the scoring interface. The post- study sur- vey questions are provided in Appendix A. The MicroNyx platform ensured that each participant com- pleted one traditional PSG session and one self- applied PSG session in randomised order, followed by a one- week break, after which they repeated the process with a second pair of ses- sions. Recommendations were presented in only one of the two traditional sessions and one of the two self- applied sessions, ensuring that if recommendations were provided in the first traditional session, they would not be shown in the second tra- ditional session, and the same rule applied to the self- applied sessions. Recommendations are covered in more detail in the following section. 2.7 | Analysis Data analysis was performed using Python and R, with tools chosen to suit the specific requirements of each test. In Python, the Pandas library (McKinney 2010) was used for data prepara- tion and organisation. For simpler single- variable analyses, the SciPy library (Virtanen et al. 2020) was employed to conduct hypothesis testing through paired t- tests, assessing significant differences in decision- making time and accuracies under dif- ferent conditions, assuming normality. When the normality assumption was unmet, the Mann–Whitney test (Mann and Whitney 1947) was applied as a non- parametric alternative. A significance level of alpha=0.05 was used throughout. To inves- tigate the relationship between scoring variables and decision- making time, the R programming language (R Core Team 2023) and the ARTool library (Kay et al. 2025) were utilised to per- form an Aligned- Rank- Transform Analysis of Variance (ART ANOVA). ART ANOVA was chosen for its ability to accommo- date the continuous nature of the decision- making time along- side categorical predictors such as recommendation correctness, presentation style and PSG type, which do not satisfy the as- sumptions of standard ANOVA. A generalised linear model was applied to analyse scoring accuracy, accounting for the binomial distribution of the dependent variable. This approach enabled the evaluation of categorical predictors' main effects and inter- actions, such as recommendation correctness, presentation style and PSG type, while respecting the constraints of binary data. FIGURE 4 | Self- applied polysomnography in the MicroNyx scoring interface.


---

Full profile (text): https://spock.is/llms

Full profile (JSON): https://spock.is/llms/json

Static profile:      https://spock.is/llms.txt