This cross-sectional study examined cognitive subtypes and influential factors in HIV-positive (HIV+) adults.
Two-step cluster analysis was conducted on a neurocognitive test battery in a sample (N = 78) of adults and older adults with HIV (Mage = 46.1). Next, cognitive, functional, and mental and physical health differences were compared between the HIV+ clusters and an HIV- reference group (N = 84; Mage = 47.9).
A two-cluster solution emerged, with a lower performing cluster exhibiting poorer performance across all domains except psychomotor speed, and a “normal” cluster displaying similar performance as the HIV- group. The most influential factors to classification in the lower performing cluster were older age and presence of stroke and hypertension. There were trends for longer duration of HIV-infection, higher unemployment rates, and greater prevalence of Hepatitis C co-infection in the lower performing cluster.
These findings suggest that there are not unique cognitive subtypes in HIV, but rather a subset of individuals who exhibit globally normal performance and those with below average performance. Older age and the related cardiovascular comorbidities of both aging and HIV medications may be key influential factors to variability in neurocognitive functioning in this population and thus should be considered in future studies. Implications for research and practice are provided.
Academic Editor: Duy Quang Trinh, Nihon University School of Medicine. Division of Microbiology, Department of Pathology and Microbiology.
Checked for plagiarism: Yes
Review by: Single-blind
Copyright © 2014 Pariya L. Fazeli,et al.
The authors have declared that no competing interests exist.
Several studies have provided evidence of lower cognitive performance for HIV-positive (HIV+) individuals compared to their HIV-negative (HIV-) counterparts in the domains of psychomotor functioning, attention, processing speed, executive functioning, and memory, reflecting a pattern of dysfunction of frontal-subcortical circuitry1, 2, 3, 4, 5, 6. These cognitive domains are also subject to declines in normal aging7. Given that cognitive abilities underlie performance of many everyday activities in individuals with HIV, such as medication management8 and driving9, examining neurocognitive impairment (NCI) in HIV is particularly important. Fortunately, the incidence of HIV-associated dementia (HAD) has decreased significantly in the era of highly active antiretroviral therapy (HAART). Yet, the incidence and prevalence of milder forms of HIV-associated neurocognitive disorders (HAND) is increasing despite potent antiretroviral therapy. One large cohort study reported that 52% of the sample had some form of HAND, with the most prevalent being asymptomatic impairment (i.e., no interference with daily functioning)10.
Yet, there is much heterogeneity in the literature on the cognitive patterns among HIV+ individuals and factors that may account for such patterns. There are various co-factors such as older age, low education level (e.g., cognitive reserve), depressive symptoms, and HIV disease severity (e.g., nadir CD4+ count) that may put some HIV+ individuals at a higher risk for poorer cognitive performance. Thus, when examining cognitive functioning in HIV, it is necessary to include these factors in order to understand the unique contribution of HIV to cognition in the context of co-factors. Additionally, it is ideal to have an HIV- reference group that is demographically similar to the HIV+ sample for more accurate comparisons. Given that some individuals with HIV may have no cognitive deficits and that there are vast individual differences in co-factors that may affect performance, examining cognitive profiles in adults with HIV is an important area for research. Using cluster analysis to identify subgroups with varying patterns of performance within HIV+ samples may provide a useful alternative to traditional comparisons of group means in HIV+ individuals as a whole, which may obscure detection of these meaningful subgroups that share similar patterns of performance and composition of co-factors.
There have been three known studies to examine cognitive subtypes in HIV. A study by van Gorp and colleagues5 examined cognitive subtypes in HIV+ males (Mage = 38.9) and yielded a three-cluster solution, with a cognitively normal cluster (39% of the sample), a cluster defined as depressed with psychomotor slowing and lowered verbal memory (28%), and a cluster with lowered overall cognitive performance and normal mood (33%). The clusters did not differ on current CD4+ count, but did differ on age, education, and HIV symptom status. Another cluster analytic study by Lojek and Bornstein1 examined patterns of cognitive functioning in HIV+ (Mage = 34.3) and HIV- (Mage = 33.1) men and yielded the following four clusters: those with psychomotor speed dysfunction (7.4% of the sample); those with memory and learning dysfunction (29.6%); those with all cognitive domains affected (10.4%); and those with no cognitive deficits or subclinical deterioration (50.6%). The clusters did not differ on level of anxiety and depression, age, current CD4+ count, or type of anti-HIV medication, but did differ on education and HIV symptom status. A final cluster analysis study by Dawes and colleagues11 of cognitive subtypes in adults with HIV (Mage = 40.68) yielded six clusters or profiles. In contrast to the two aforementioned cluster analytic studies, this study used ipsative scoring on the cognitive factors to define clusters based on pattern (not overall level) of performance. No cluster differences were found for age, education, gender, AIDS status, current or nadir CD4+ count, viral load, HAART status, Hepatitis C co-infection, subjective cognitive complaints, or current rates of major depression or substance use disorders. However, the methodology used in the cluster analysis by Dawes and colleagues yielded results that may be cumbersome to interpret as they are relative only to pattern, rather than level of performance, which may be important when the goal is isolating those with NCI. Also, this approach to cluster analysis yielded twice the number of clusters as the other two studies, which may be difficult to interpret due to lack of parsimony.
Overall, the three cluster analytic studies yielded some consistent findings. First, they suggest that there is not one prototypical pattern of NCI in HIV, with some individuals having no cognitive deficits, some exhibiting declines in specific domains, while others may have global impairment. Second, there are several co-factors such as education, age, and HIV symptom status that may influence cognitive performance and patterns. Unfortunately, these studies have several limitations, including relatively young adult samples (i.e., the highest mean age being 40.68 years in the study by Dawes and colleagues), two of the studies used only male samples, the van Gorp and colleagues study occurred before HAART was developed, and only one of the studies examined nadir CD4+ count. Additionally, only the Lojek and Borstein study explicitly compared their HIV+ clusters to an HIV- reference group. Thus, there is a need for more cluster analyses of the cognitive subtypes in HIV using older samples, including both males and females, including an HIV- reference group, and examining more co-factors. Specifically, co-factors that should be examined include substance use, depressive symptoms, Hepatitis C co-infection, medical comorbidites, HAART adherence, and nadir CD4+ count.
The purpose of the current study was to examine cognitive subtypes and influential correlates among HIV-infected individuals. Specifically, the goal of aim 1 was to examine cognitive subtypes in a sample of HIV+ adults and older adults using cluster analysis. The goal of aim 2 was to confirm the validity of the cognitive clusters by examining the differences in performance on cognitive and everyday functioning measures between the HIV+ clusters and an HIV- reference group. The goal of aim 3 was to determine whether the clusters differed on HIV-specific co-factors (e.g., nadir CD4+ count) and to compare the clusters to an HIV- reference group on non-HIV specific co-factors (e.g., depressive symptoms). Finally, the goal of aim 4 was to examine the prevalence of psychometrically defined NCI in the overall HIV+ sample, and stratified by the HIV+ clusters.
Three-hundred and forty-seven adults recruited from the Birmingham, Alabama, metropolitan area were telephone screened for the original cross-sectional parent study. The current study uses data from those eligible participants that did not meet exclusion criteria and thus completed the parent study, as described below. HIV+ participants were recruited from a university-based HIV/AIDS community clinic with flyers and brochures. HIV- participants were recruited from flyers, brochures, university newspaper advertisements, and word-of-mouth. Interested participants called the research center, and a telephone screening interview was conducted to determine eligibility. HIV+ participants must have known about their HIV diagnosis for at least one year in order to eliminate the potential confounds of reactive anxiety and depression that may accompany an initial HIV diagnosis. Additional exclusion criteria for the entire sample included being homeless, pregnant, blind, deaf, having a developmental disability, undergoing chemotherapy or radiation, not being proficient in speaking and reading English, past brain injury involving a loss of consciousness for longer than 30 minutes, or having a severe neurological condition (e.g., schizophrenia, bipolar disorder, HIV encephalopathy, dementia). In addition to the self-report information on the presence of these neurological co-morbidities, this information was confirmed with the HIV clinic medical charts for the HIV+ participants. After excluding those who met the exclusion criteria, 78 HIV+ participants (Mage = 46.61; 24% women) and 84 HIV- participants (Mage = 47.93; 60% women) remained. The HIV- participants were included in the current study in order to have a reference group for comparisons between HIV+ clusters and were thus recruited by the parent study to be demographically similar to the HIV+ group with regard to age and education.
All participants completed a 2½ hour battery consisting of demographic, mental and physical health, cognitive, and functional measures administered by experienced testers. Participants were compensated $50 for their time. For the HIV+ participants only, the health questionnaire included additional questions regarding self-reported current CD4+ count and plasma viral load. Since participants were recruited from the university HIV/AIDS clinic, computerized chart extraction of their most recent laboratory values for current and nadir CD4+ count and viral load was also available. Clinic values were used in analyses rather than self-reported values, except when missing as stated below. For 75 participants who had both self-reported and clinic values for their current CD4+ count, there was a high level of agreement (r = .73, p < .001). Thus, current CD4+ count values for the three participants whose values were missing from the clinic were substituted using their self-reported values. For 32 participants who had both self-reported and clinic values for plasma viral load, there was a low level of agreement (r = .01 p = .92); thus, only clinic values of plasma viral load were used and for the four cases that were missing this information, substitution was not deemed appropriate. Clinic values for nadir CD4+ count were available for 70 participants; since there was a reasonable correlation between nadir CD4+ count and current clinic values for CD4+ count (r = .67, p < .001), for the remaining eight cases, their current CD4+ count was used to substitute this missing value in subsequent analyses. The HIV+ participants also completed an HIV medication adherence questionnaire (Simplified Medication Adherence Questionnaire12), with higher scores reflecting greater nonadherence.
All participants completed a stressful life events questionnaire (Social Readjustment Scale13), a mood questionnaire (Profile of Mood States14), a substance use questionnaire (Addiction Severity Index15), and a health questionnaire (Cardiovascular Health Study16). Higher scores reflect greater number of stressful life events, higher depressive symptoms, and greater substance use. For the health questionnaire participants self-reported whether they had ever received a diagnosis for several medical conditions (e.g., hypertension; mood problems such as depression or anxiety) along with the medications they were currently taking.
Cognitive and Everyday Functioning Test Battery
References are provided for those cognitive measures that are traditionally used in the HIV and aging literature, while detailed descriptions are provided for those additional cognitive measures for which readers may be less familiar (i.e., UFOV and CRT), as well as for the everyday functioning measures.
Visual Attention.Useful Field of View Test® (UFOV).The UFOV® test is a computerized measure of visual attention and processing speed17. There are four subtests that increase in difficulty as participants progress through the test. In each subtest participants must attend to central and peripheral (or both) visual stimuli and the display duration (17-500ms) of the stimuli become shorter, and thus more difficult, as they progress. This allows for quantification of processing speed by using display duration threshold as the score. Using a double-staircase method, scores are generated for each subtest which reflects the display duration in which 75% accuracy has been achieved. These scores were combined to create a total UFOV® score, with lower scores indicating fewer milliseconds needed to correctly perceive the stimuli, and thus better processing speed.
Reaction Time.Complex Reaction Time Test (CRT).The CRT test is a computerized measure of everyday processing speed and reaction time18, 19. Participants were presented with several road signs (i.e., left and right turn arrows, pedestrian, and bicycle) and instructed to react as quickly as possible in a specific way (either a single click or moving the mouse right or left). There are two trials of 12 presentations (the first presents three signs at a time, while the second presents six). Participants’ average reaction time in seconds was used as the score for this test, with lower scores indicative of faster processing speed.
Processing Speed.Letter and Pattern Comparison task20.
Psychomotor Speed.Finger Tapping Test21.
Executive Function.Wisconsin Card Sorting Test (WCST) Computer Version 422, 23.
Memory.Hopkins Verbal Learning Test (HVLT)24.
Speed of Everyday Functioning.Timed Instrumental Activities of Daily Living (TIADL). The TIADL is a measure of everyday functioning25. It measures both the speed and accuracy in which five typical everyday activities are completed (e.g., finding two food items on a shelf of food, using coins to count out correct change, finding a telephone number in a phone book). The amount of time (seconds) necessary to complete each task is used as the score. If the task is not completed within the time limit (e.g., two minutes), the task is then terminated and the participant is given the maximum time limit as the score for that task. If the task is completed within the pre-set time limit but is performed incorrectly, a time penalty is added. This penalty is equal to one standard deviation of time that is derived from the scores of those who performed that task within the time limit. The final scores are transformed into a z-score for each of the five tasks and averaged in order to provide a TIADL composite score; this standardization ensures that the tasks are equally weighted. Since z-scores are used, composite scores can be reflected in negative and positive coefficients; lower composite scores indicate better performance on this test. This measure has evidence of good test-retest reliability (r = .64)25.
Complex Everyday Functioning.Observed Tasks of Daily Living (OTDL).This measure is composed of 28 observational tasks that simulate complex and instrumental activities of daily living that require inferential thinking and have observable elements allowing objective scoring of performance26. The tasks include medication, telephone, and financial-related activities. Participants are given stimulus items (e.g., medicine bottles) and a card with a question on it for each activity. This is not a timed task; rather, accuracy is recorded (i.e., correct or incorrect), and whether or not a prompt was needed. Total scores are calculated based on accuracy and use of prompts; higher scores reflect better everyday functioning. The mean kappa across tasks for all three domains is 0.93.
All statistics were conducted using SPSS version 20. Descriptives and group differences on the demographic, mental and physical health, cognitive, and functional variables were conducted between the HIV- and HIV+ samples using ANOVA and chi-square analyses. In order to examine cognitive subtypes in a sample of adults with HIV, cluster analysis was employed. Formann27 suggested a sample size of no less than 2k (k = number of variables). With this relatively small sample size (N = 78), a maximum of six variables could be entered into the cluster analysis. As highly correlated variables are not recommended for cluster analysis, we first examined whether there was any substantial multicollinearity (r > 0.70) among the six cognitive measures (UFOV®, CRT, Letter and Pattern Comparison, WCST (percentage of correct responses), HVLT, and the Finger Tapping Test).
The Two-Step method, a newer clustering approach that is a variant of the Hierarchical clustering technique28 was chosen for the current study. All cognitive measures were standardized to z-scores by default in the Two-Step procedure. The advantage to the Two-Step clustering method is that it provides a measure of the most appropriate number of clusters using the Schwarz Bayesian Information Criterion, a general measure of the overall fit of a solution based upon the mean square error. This clustering method is preferred when the most appropriate number of clusters to fit the data is not known prior to the clustering procedure. In order to demonstrate the stability of the cluster solution yielded from the Two-Step method, a Hierarchical and a K-Means cluster analysis were also employed using the number of clusters yielded from the Two-Step approach.
Comparisons were conducted between the HIV+ clusters and the HIV- reference group on the six cognitive measures used to form the clusters as well as the two measures of everyday functioning. These measures included: UFOV®, CRT, Letter and Pattern Comparison, WCST (percentage of correct responses and number of categories completed), HVLT, the Finger Tapping Test, TIADL, and OTDL. MANOVA was used with Bonferonni’s post-hoc tests for follow-comparisons.
To determine whether clusters differed on HIV-specific co-factors (i.e., years with HIV, medication adherence, current and nadir CD4+ count, and plasma viral load) and to compare the clusters to an HIV- reference group on non-HIV specific co-factors (i.e., age, education, income, race, sexual orientation, gender, depression, stressful life events, medical co-morbidities, medications, employment status, Hepatitis C co-infection, and substance use), MANOVA was used for continuous variables and chi-square analyses were used for dichotomous variables. In addition to examining the total number of medical co-morbidities that may affect cognition (i.e., self-reported mood problems, Diabetes, hypertension, stroke, and hepatitis C), each of these conditions was also examined separately.
To further examine the validity of the cluster solution, the clusters were examined for psychometrically defined NCI. Using the mean and standard deviations of the demographically similar HIV- reference group for each of the six cognitive measures, z-scores were created for the HIV+ group. Participants whose performance was one or more standard deviations in the impaired direction for two or more measures were classified as having psychometrically defined NCI. Percentages were calculated for the composition of impairment in the clusters as well as for the total HIV+ sample.
One data point was missing for the following cognitive tests: the Finger Tapping Test, CRT, UFOV®, and WCST. Based on the remaining cognitive scores, linear regression was used to impute these missing values. Preliminary analyses confirmed that the HIV+ group and the HIV- reference group were demographically similar (Table 1). There were no significant differences between the groups on age, percentage over age 50, race, income, education, depression (i.e., Profile of Mood States), stressful life events, and drug and alcohol use. The HIV- group had a significantly higher proportion of heterosexuals and currently employed participants. The HIV+ group had significantly more males and individuals with Hepatitis C. The HIV+ group reported more medical conditions and prescribed medications, as would be expected in this clinical population. Despite the comparability of the HIV+ and HIV- groups on the Profile of Mood States, the HIV+ group had a significantly higher frequency of self-reported mood problems (depression or anxiety; self-reported from the health questionnaire) than the HIV- group. Regarding the cognitive and functional measures, the HIV+ group performed significantly worse than the HIV- group on the CRT, Letter and Pattern Comparison, and the TIADL, while a trend emerged on the UFOV Test.Table 1. Group Differences Between the HIV+ and HIV- Samples (N = 162)
|HIV+(n = 78)||HIV-(n = 84)|
|Variable||n (%)||M (SD)||n (%)||M (SD)||p-value|
|Age||46.61 (10.40)||47.93 (13.06)||0.48|
|No. Over Age 50 (%)||31 (40%)||41 (49%)||0.25|
|No. Men (%)‡||59 (76%)||33 (39%)||0.00|
|No. Heterosexuals (%)‡||39 (50%)||78 (93%)||0.00|
|No. Caucasians (%)*||48 (62%)||55 (65%)||0.73|
|No. Working (%)‡||12 (15%)||35 (42%)||0.00|
|Income||1.74 (1.35)||1.98 (1.53)||0.31|
|Education (years)||12.77 (2.48)||12.79 (1.68)||0.96|
|No. Med. Conditions‡||1.59 (1.14)||1.06 (0.99)||0.00|
|No. w/ Hepatitis C (%)‡||26 (33%)||6 (7%)||0.00|
|No. w/ Mood Prob. (%)†||44 (56%)||33 (39%)||0.03|
|No. w/ Stroke (%)||7 (9%)||5 (6%)||0.46|
|No. w/ Hypertension (%)||38 (49%)||32 (38%)||0.17|
|No. w/ Diabetes (%)||9 (12%)||10 (12%)||0.94|
|No. Medications‡||4.83 (3.39)||2.18 (2.74)||0.00|
|POMS Total||35.47 (40.29)||28.26 (37.89)||0.24|
|POMS-Positive||17.73 (6.68)||19.27 (6.52)||0.14|
|POMS-Negative||53.21 (36.55)||47.54 (34.87)||0.31|
|Stressful Life Events||268.29 (139.58)||238.51 (164.16)||0.22|
|ASI - Alcohol Use||0.23 (0.60)||0.24 (0.45)||0.90|
|ASI - Drug Use||0.03 (0.07)||0.02 (0.04)||0.12|
|UFOV® Test||737.73 (361.48)||638.45 (334.64)||0.07|
|CRT†||1.93 (0.56)||1.75 (0.47)||0.02|
|Letter & Pattern†||76.67 (17.36)||82.75 (17.28)||0.03|
|WCST % Correct||50.54 (18.26)||54.35 (18.84)||0.19|
|WCST Cat. Completed||2.52 (2.20)||3.07 (2.25)||0.12|
|Finger Tapping Test||50.52 (7.73)||48.63 (8.27)||0.14|
|HVLT||23.53 (6.28)||24.12 (6.21)||0.55|
|TIADL†||0.65 (3.44)||-0.61 (2.65)||0.01|
|OTDL||68.10 (7.59)||69.71 (7.38)||0.17|
Descriptive analyses for the HIV+ sample revealed that 87% of the sample was currently taking a HAART regimen (Table 2). Fifteen percent of the sample had a current CD4+ count below 200 (indicative of AIDS). Forty-two percent of the sample had a nadir CD4+ count below 200. Finally, 38% of the sample had an undetectable plasma viral load.Table 2. Descriptives for HIV+ Sample on HIV-Related Variables (N = 78)
|Variable||n (%)||M (SD)||Range|
|Years with HIV||12.93 (7.34)||1.00 - 26.10|
|No. Taking ART (%)||68 (87%)|
|Medication Adherence||3.19 (4.31)||0.00 - 17.00|
|Current CD4+ count||471.30 (274.40)||11.00 - 1,140.00|
|Nadir CD4+ count||276.39 (236.60)||1.00 - 1,037.00|
|Current Viral Load||14,780.82 (67,501.02)||48.00 - 549,000.00|
|No. with Current CD4+ count < 200 (%)*||12 (15%)|
|No. with Nadir CD4+ count < 200 (%)*||33 (42%)|
|No. with Undetectable Viral Load (%)||28 (38%)|
Across all of the cognitive measures (UFOV®, CRT, Letter and Pattern Comparison, Finger Tapping Test, WCST, HVLT), correlation coefficients did not exceed 0.51, indicating that there was no substantial multicollinearity (Table 3), suggesting that each of the measures represented relatively different constructs. To examine whether the outcome of the cluster analysis would be affected by including different combinations of variables, the cluster analysis was performed including different combinations of the variables (i.e., including only three, four, and five cognitive measures). In no situation was the cluster solution (i.e., number of clusters yielded) any different than the original analysis including all six measures. Thus, all six measures were entered into the final cluster analysis. We also conducted a factor analysis on the cognitive measures and the results of the cluster analysis remained the same.Table 3. Correlations for Cognitive Measures (N = 78)
|1.Useful Field of View||1.00|
|2.Complex Reaction Time||0.51**||1.00|
|3.Letter & Pattern Comparison||-0.44**||-0.44**||1.00|
|4.WCST Percentage Correct||-0.32**||-0.31**||0.33**||1.00|
|5.Finger Tapping Test||-0.25*||-0.13||0.33**||-0.03||1.00|
Results of the Two-Step cluster analysis of the HIV+ sample yielded a two cluster solution as the most appropriate, as determined by the lowest Schwarz Bayesian Information Criterion and Schwarz Bayesian Information Criterion change values (357.18 and -16.49, respectively). Cluster 1 contained 32 participants while Cluster 2 contained 46. To examine the stability and consistency of this cluster solution, a K-Means and Hierarchical cluster analysis were performed with a specified solution of two clusters and showed that 83% of the participants in the K-Means analysis, and 90% of those in the Hierarchical analysis were correctly classified in the two clusters yielded from the initial Two-Step procedure.
Results of the MANOVA comparing cognitive and functional differences between the HIV+ clusters and the HIV- reference group revealed that Cluster 1 performed significantly worse than Cluster 2 and the HIV- reference group on each measure except for the Finger Tapping Test, for which there were no group differences. Cluster 2 performed similarly to the HIV- group on every measure except the HVLT, where Cluster 2 actually had significantly better performance than the HIV- group. Similarly, for the OTDL and TIADL, Cluster 1 performed significantly worse than both Cluster 2 and the HIV- group, while Cluster 2 and the HIV- group did not significantly differ (Table 4).Table 4. Cognitive and Functional Test Scores of the HIV+ Clusters and the HIV- Reference Group (Total N = 162)
|Cluster 1(n = 32)||Cluster 2(n = 46)||HIV- Group(n = 84)|
|UFOV® Test||1039.78||279.19||527.61||244.72||638.45||334.64||< .0001a,b|
|Complex Reaction Time||2.36||0.53||1.64||0.34||1.75||0.47||< .0001a,b|
|Letter & Pattern Comparison||67.81||16.83||82.83||15.04||82.75||17.28||< .0001a,b|
|WCST Percent Correct||40.83||15.83||57.30||16.85||54.35||18.84||< .0001a,b|
|WCST Categories Completed||1.19||1.67||3.45||2.06||3.07||2.25||< .0001a,b|
|Finger Tapping Test||49.08||7.91||51.52||7.52||48.63||8.27||ns|
Results of the analyses examining influential factors to cluster membership revealed that Cluster 1 was significantly older than Cluster 2. Further, Cluster 1 and the HIV- group had a significantly higher percentage of participants over age 50 than Cluster 2. There was a trend for current employment and Hepatitis C co-infection between Clusters 1 and 2, with a trend towards Cluster 1 having fewer participants who were employed and a higher prevalence of Hepatitis C. Cluster 1 reported significantly more medical conditions than Cluster 2 and the HIV- group. Of these conditions, Cluster 1 had a significantly higher frequency of both stroke and hypertension than Cluster 2 and the HIV- group. There were no significant differences between Clusters 1 and 2 and the HIV- group on proportion of Caucasians, income, education, mood, stressful life events, alcohol use, and drug use (Table 5). Clusters 1 and 2 did not significantly differ on any of the HIV-specific variables; however, there was a trend for years with HIV, with those in Cluster 1 on average having a longer diagnosis of HIV (Table 6).Table 5. Demographic and Mental and Physical Health Differences of the HIV+ Clusters and the HIV- Reference Group (Total N = 162)
|Cluster 1(n = 32)||Cluster 2(n = 46)||HIV- Group(n = 84)|
|No. Over Age 50 (%)||19 (59%)||12 (26%)||41 (49%)||< .01a, c|
|No. Men (%)||22 (69%)||37 (80%)||33 (39%)||< .05b, c|
|No. Heterosexuals (%)||20 (63%)||19 (41%)||78 (93%)||< .05b, c|
|No. Caucasians* (%)||8 (25%)||21 (46%)||29 (36%)||ns|
|No. Working (%)||2 (6%)||10 (22%)||35 (42%)||< .001b, c, †|
|No. Med. Conditions||1.94||1.24||1.34||1.02||1.06||0.99||< .01a, b|
|No. w/ Hepatitis C (%)||14 (44%)||12 (26%)||6 (7%)||< .001b, c, †|
|No. w/ Mood Prob. (%)||18 (56%)||26 (57%)||33 (39%)||ns|
|No. w/ Stroke (%)||6 (19%)||1 (2%)||5 (6%)||< .05a, b|
|No. w/ Hypertension (%)||20 (63%)||18 (39%)||32 (38%)||< .05a, b|
|No. w/ Diabetes (%)||4 (13%)||5 (11%)||10 (12%)||ns|
|No. Medications||5.25||3.85||4.54||3.04||2.18||2.74||< .001b, c|
|Stressful Life Events||263.56||151.71||271.59||132.10||238.51||164.16||ns|
|Cluster 1(n = 32)||Cluster 2(n = 46)|
|Variable||n (%)||M||SD||n (%)||M||SD||p|
|Years with HIV||14.68||7.91||11.72||6.73||0.08|
|No. Taking ART (%)||27 (84%)||41 (91%)||0.39|
|Current CD4+ count||498.50||247.20||452.37||293.00||0.47|
|Nadir CD4+ count||329.72||225.33||239.28||239.54||0.10|
|Current Viral Load||5,395.70||17,159.16||21,179.77||86,216.81||0.33|
|No. with Current CD4+ count < 200 (%)*||4 (13%)||8 (17%)||0.40|
|No. with Nadir CD4+ count < 200 (%)*||11 (34%)||22 (48%)||0.17|
|No. with Undetectable Viral Load (%)||14 (47%)||14(32%)||0.15|
Results for aim four revealed that 91% (n = 29) of Cluster 1 participants were classified with psychometrically defined-NCI, compared to 17% (n = 8) of Cluster 2. Further, in Cluster 2, of those who were classified as impaired, all but one of these participants (who exhibited lower performance on three tests) only exhibited lowered performance in two tests, while Cluster 1 contained participants who performed worse on between three and six measures. When considering the HIV+ sample as a whole regardless of cluster membership, 47% (n = 37) of the sample was classified with psychometrically defined-NCI.
Using cluster analysis in a sample of adults and older adults with HIV, two cognitive clusters emerged: a lower performing cluster and a cognitively “normal” cluster who was comparable to an HIV- reference group, suggesting successful cognitive aging in a subset of HIV+ adults. When comparing the HIV+ clusters to the HIV- reference group across cognitive and functional measures, the validity of the two-cluster solution was confirmed, with generally lower performance in Cluster 1 relative to Cluster 2 and the HIV- reference group. The lack of a significant difference for the Finger Tapping Test suggests that psychomotor speed may be spared in the face of well-controlled HIV. The finding that the “normal” cluster had better performance on the HVLT, while statistically significant, the fact that on average these “normal” participants recalled two more words than the HIV- group may not have everyday clinical implications.
The two-cluster solution we found is both in parallel and inconsistent to the literature. It is in parallel with the literature suggesting that there is a subset of HIV+ individuals with global lower performance and a subset with global higher performance (“normals”). It is incongruent with prior studies that have yielded cluster solutions of three or more clusters, with some clusters defined as having relative decrements in specific cognitive domains only (e.g., psychomotor only). There are two major explanations for this. First, these prior cluster analytic studies had much larger sample sizes, which may have made it possible to detect these distinct subgroups in the data. Second, as these prior studies used factor analysis, their cognitive “factors” were forced to be orthogonal (uncorrelated), making it more likely to detect distinct subgroups rather than only groups with overall lower/higher performance. However, even when post-hoc factor analysis was employed in our study the results remained the same.
Age emerged as an influential factor to cluster membership suggesting that older age itself may account for the lower performance in Cluster 1. While the “normal” cluster was not significantly younger than the HIV- group, they were about four years younger on average, which may have been, in part, an explanation for their comparability to the HIV- group. This is further implicated by the finding that the lower performing cluster and the HIV- group had a significantly higher proportion of individuals over age 50 (59% and 49%, respectively) than the “normal” cluster (26%) despite not having significantly different mean ages. The higher prevalence of hypertension and prior stroke in the lower performing cluster than the “normal” cluster and the HIV- group highlights the importance of considering comorbid medical conditions in the context of neurocognition in HIV, as such cardiovascular conditions are common side effects of both aging and HIV medications. The trend for Hepatitis C prevalence and unemployment differing between the two HIV+ clusters (with the lower performance cluster having fewer employed individuals and more individuals with Hepatitis C) is interesting and suggests that Hepatitis C co-infection and unemployment may play a role in their poorer performance. Regarding employment, the direction of this relationship is unknown (i.e., are they performing worse because they are not employed and thus experiencing a lack of mental stimulation, or are they not employed because of initial cognitive problems?); however, there may be a bidirectional relationship. Causal inferences cannot be made with these cross-sectional analyses. The association of total number of reported medical conditions with cluster membership may have been driven by the independent effects of hypertension and stroke, but nonetheless implies the importance of treating medical conditions that may affect cognition in individuals with HIV. For number of prescribed medications, the finding that the HIV+ clusters both had significantly more medications than the HIV- group was not surprising as this difference is typical of HIV+ samples and is expected given the pill regimens of HAART.
The finding that there were no significant differences between the HIV+ clusters on current CD4+ count was congruent with prior cluster analytic studies. However it was surprising that nadir CD4+ lymphocyte count (an index of disease severity) was not related to cluster membership. This may be due to the majority of the sample being relatively healthy and AIDS-free. The finding that being prescribed to HIV medications and medication adherence were not related to cognitive performance was also surprising; however this sample contained a large majority of individuals who were prescribed HIV medications and were largely adherent to these medications, limiting the variability with which to examine the effect of these variables. While not statistically significant, the trend for years with HIV, with those in the lower performing cluster having been diagnosed with HIV for about three years more on average than the “normal” cluster may suggest at first glance that individuals who have had HIV longer may be at an increased risk for cognitive declines. However, since years with HIV and age were moderately correlated (r = 0.42), it may be that age at least partially explains this relationship, as years with HIV is inherently confounded by age, making it difficult to disentangle the potential effect of length of HIV disease on neurocognition in long-term survivors.
The final analysis confirmed the validity of the cluster solution, showing that a majority of the lower performing cluster participants were classified with psychometrically defined NCI, and a majority of the “normal” cluster were defined as cognitively “normal” compared to the HIV- reference group. The finding that 47% of the HIV+ sample exhibited at least subtle NCI is congruent with the findings on the prevalence of NCI in the HAART era10. These cognitive test scores were raw, uncorrected for age, which should be noted when interpreting psychometrically defined NCI, which was relative to our sample means rather than population normative data. Albeit, this was deemed appropriate in the current study given the comparability of our HIV- and HIV+ groups on demographic factors.
Cognitive declines may not necessarily occur in all HIV+ persons. Age and the associated co-morbid conditions of both aging and HIV should continue to be considered when examining neurocognition in those aging with HIV in both clinical and research settings. Clinicians and researchers should be aware of potential cognitive declines in adults with HIV, even if these declines are subtle. This study highlights the importance of using a demographically similar HIV- reference group when examining cognitive dysfunction in HIV+ samples in order to avoid overestimation of cognitive dysfunction. Additionally, the finding that 47% of the HIV+ sample had some form of NCI was in parallel with the current HIV literature, and thus underscores that although HAD is decreasing, more subtle cognitive decrements are still prevalent and should be taken seriously and monitored by individuals with HIV as well as their healthcare team.
Limitations and Directions for Future Research
Significant limitations of the current study include the relatively small sample size and cross-sectional design. Further, only self-report questionnaires of mood and current (i.e., 30-day) substance use were administered, thus, future studies should utilize structured clinical interviews to ascertain both substance use and psychiatric disorders. Future studies should examine cognitive subtypes in HIV using very large sample sizes (i.e., thousands of participants) for optimal performance of the cluster analysis technique. In cluster analysis, the larger the sample, the more variables that can be included, thus larger sample sizes may allow for a greater number of distinct clusters to be discovered. Additionally, more studies are needed that include older samples (i.e., aged 50 and above). While participants in this age group may have been scarce in years prior to the advent of HAART, with the increase in the prevalence and incidence of HIV in adults over age 50, these individuals will be an important population to examine in the coming years. Also, longitudinal studies are needed that examine the trajectory of cognitive functioning and cognitive change in adults with HIV as they age into older adulthood.
In addition to the need for future research examining cognitive declines associated with HIV, intervention strategies to ameliorate such cognitive declines are needed. For example, research in the gerontological literature has suggested that computerized cognitive remediation therapy may be an effective intervention strategy to help improve or maintain cognition, especially in the domain of processing speed18. Pilot research has also utilized this technique in a sample of adults with HIV and it was found to be effective in improving processing speed and performance of speeded everyday functioning compared to a no-contact control group29. Additionally, future research is needed to examine the efficacy of preventive strategies to avoid cognitive dysfunction in HIV. For example, Vance and colleagues30 have posited the concept of theoretical “cognitive prescriptions” which are individualized behavioral plans given by clinicians to help promote habits that may increase positive neuroplasticity (e.g., healthy diet, exercise, intellectual stimulation), and reduce habits that may increase negative neuroplasticity (e.g., substance abuse, depression). While these healthy and unhealthy behaviors are commonly acknowledged by individuals, many may not be aware of the potential relationship of these lifestyle habits to cognition and thus performance of everyday activities. Thus, educating individuals with HIV (especially those with subjective cognitive complaints) about this relationship may make them more inclined to adjust their behaviors to promote better cognitive functioning.
This article was written with support from an R03 from the National Institute of Mental Health “Chronicity of HIV and Aging on Neuropsychological and Everyday Performance” (R03MH076642-01A2) and the University of Alabama at Birmingham (UAB) Edward R. Roybal Center for Translational Research on Aging and Mobility Project (Grant No. 2 P30 AG022838-06).
- 1.Lojek E, R A Borstein. (2005) The stability of neurocognitive patterns in HIV-infected men: Classification considerations. , J Clin Exp Neuropsyc 27, 665-682.
- 2.Murji S, Rourke S, Donders B, Carter J, Shore S L et al. (2003) Theoretically derived CVLT subtypes in HIV-1 infection: Internal and external validation. , J Int Neuropsych Soc 9, 1-16.
- 3.B A Navia, B D Jordan, R W Price. (1986) The AIDS dementia complex: I. Clinical features. , Ann Neurol 19, 517-524.
- 4.Reger M, Welsh R, Razani J, D J Martin, K B Boone. (2002) A meta-analysis of the neuropsychological sequelae of HIV infection. , J Int Neuropsych Soc 8, 410-424.
- 5.Gorp W G van, Hinkin C, Satz P, E N Miller, Weisman J. (1993) Subtypes of HIV-related neuropsychological functioning: A cluster analysis approach. , Neuropsychology 7(1), 62-72.
- 6.D E Vance, P L Fazeli, C A Gakumo. (2013) The impact of neuropsychological performance on everyday functioning between older and younger adults with and without HIV. , J Assoc Nurse AIDS C 24(2), 112-125.
- 7.Craik F I M, T A Salthouse. (2000) . Handbook of aging and cognition (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates .
- 8.C H Hinkin, D J Hardy, K I Mason, S A Castellon, R S Durvasula. (2004) Medication adherence in HIV-infected adults: Effect of patient age, cognitive status, and substance abuse. AIDS,18(Suppl1): S19-S25.
- 9.T D Marcotte, Wolfson T, T J Rosenthal, R K Heaton, Gonzalez R. (2004) A multimodal assessment of driving performance in HIV infection. , Neurology 63, 1417-1422.
- 10.R K Heaton, D B Clifford, Jr Franklin, R D, S P Woods et al. (2010) HIV-associated neurocognitive disorders persist in the era of potent antiretroviral therapy. , Neurology 75, 2087-2096.
- 11.Dawes S, Suarez P, C Y, Cherner M, T D Marcotte. (2008) Variable patterns of neuropsychological performance in HIV-1 infection. , J Clin Exp Neuropsyc 30(6), 613-626.
- 12.Knobel H, Alonso J, J L Casado, Collazos J, Gonzalez J. (2002) Validation of a simplified medication adherence questionnaire in a large cohort of HIV-infected patients:. , The GEEMA Study. AIDS 16(4), 605-613.
- 13.Holmes T H, Rahe R H. (1967) The Social Readjustment Rating Scale. , J Psychosom Res 11, 213-218.
- 14.D M McNair, Lorr M, L F Droppelman. (1992) . Profile of Mood States manual. San Diego, CA: Educational and Industrial Testing Services .
- 15.A T McLellan, Kushner H, Metzger D, Peters R, Smith I. (1992) The Fifth Edition of the Addiction Severity Index. , J Subst Abuse Treat 9(3), 199-213.
- 16. (1989) Cardiovascular Health Study.Manual of operations. Seattle,WA:University of Washington Coordinating Center.
- 17.J D Edwards, D E Vance, V G Wadley, G M Cissel, D L Roenker. (2005) Reliability and validity of the Useful Field of View test scores as administered by personal computer. , J Clin Exp Neuropsyc 27(5), 529-543.
- 18.Ball K, Berch D B, Helmers K F, Jobe J B, Leveck M D. (2002) Effects of cognitive training interventions with older adults: A randomized controlled trial. , J Am Med Assoc 288, 2271-2281.
- 19.Ball K, Owsley C. (2000) Increasing mobility and reducing accidents of older drivers. In K. W. Schaie & M. Pietrucha (Eds.), Mobility and transportation in the elderly.New York:Springer Publishing Company,Inc 213-251.
- 20.T A Salthouse. (1991) Mediation of adult age differences in cognition by reductions in working memory and speed of processing. , Psychol Sci 2(3), 179-183.
- 21.Reitan R M, Wolfson D. (1985) The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation.Tucson,AZ:Neuropsychological Press.
- 22.Heaton S K, Chelune G J, Talley J L, Kay G G, Curtiss G. (1993) Wisconsin Card Sorting Test manual: Revised and expanded. Odessa, FL: Psychological Assessment Resources.
- 23.Heaton R K. (2003) Wisconsin Card Sorting Test Computer Version 4. Lutz, FL:Psychological Assessment Resources.
- 24.Brandt J. (2001) The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms. , Clin Neuropsychol 5(2), 125-142.
- 25.Owsley C, Sloane M, McGwin GJr, Ball K. (2002) Timed instrumental activities of daily living task: Relationship to cognitive function and everyday performance assessments in older adults. , Gerontology 48(4), 254-265.
- 26.Diehl M, S L Willis, K W Schaie. (1995) Everyday problem solving in older adults: Observational assessment and cognitive correlates. , Psychol Aging 10(3), 478-491.
- 27.Formann A K. (1984) Die Latent-Class-Analyse: Einfuhrung in die Theorie und Anwendung. , Beltz, Weinheim
- 28.Chiu T, Fang D, Chen J, Wang Y, Jeris C. (2001) A robust and scalable clustering algorithm for mixed type attributes in large database environments. In:. Proceedings of the 7th ACM SIGKDD international conference in knowledge discovery and data mining, Association for Computing Machinery , San Francisco, CA 263-268.
- 29.D E Vance, P L Fazeli, L A Ross, Wadley V, Ball K K. (2012) The effect of speed of processing training on middle-aged and older adults with HIV: A pilot study. , J Assoc Nurse AIDS C 23(6), 500-510.
Cited by (30)
- 1.Duggan Michael R., Mohseni Ahooyi Taha, Parikh Vinay, Khalili Kamel, 2021, Neuromodulation of BAG co-chaperones by HIV-1 viral proteins and H2O2: implications for HIV-associated neurological disorders, Cell Death Discovery, 7(1), 10.1038/s41420-021-00424-0
- 2.Torkzaban Bahareh, Mohseni Ahooyi Taha, Duggan Michael, Amini Shohreh, Khalili Kamel, 2020, Cross-talk between lipid homeostasis and endoplasmic reticulum stress in neurodegeneration: Insights for HIV-1 associated neurocognitive disorders (HAND), Neurochemistry International, 141(), 104880, 10.1016/j.neuint.2020.104880
- 3.Calon Maëliss, Menon Kritika, Carr Andrew, Henry Roland G., Rae Caroline D., et al, 2020, Additive and Synergistic Cardiovascular Disease Risk Factors and HIV Disease Markers' Effects on White Matter Microstructure in Virally Suppressed HIV, JAIDS Journal of Acquired Immune Deficiency Syndromes, 84(5), 543, 10.1097/QAI.0000000000002390
- 4.Devlin Kathryn N., Giovannetti Tania, 2017, Heterogeneity of Neuropsychological Impairment in HIV Infection: Contributions from Mild Cognitive Impairment, Neuropsychology Review, 27(2), 101, 10.1007/s11065-017-9348-2
- 5.Paul Robert H., Cooley Sarah A., Garcia-Egan Paola M., Ances Beau M., 2018, Cognitive Performance and Frailty in Older HIV-Positive Adults, JAIDS Journal of Acquired Immune Deficiency Syndromes, 79(3), 375, 10.1097/QAI.0000000000001790
- 6.Lindayani Linlin, Sudrajat Diwa Agus, Melnawati Chanti, Anggarini Dian, 2020, Prevalence of neurocognitive impairment and its associated factors among patients with HIV in Indonesia, British Journal of Neuroscience Nursing, 16(6), 258, 10.12968/bjnn.2020.16.6.258
- 7.Kemp Harriet I., Kennedy Donna L., Vollert Jan, Davies Nicholas W.S., Scott Whitney, et al, 2021, Chronic pain and cognitive impairment: a cross-sectional study in people living with HIV, AIDS Care, (), 1, 10.1080/09540121.2021.1902934
- 8.Levine Andrew J., Soontornniyomkij Virawudh, Achim Cristian L., Masliah Eliezer, Gelman Benjamin B., et al, 2016, Multilevel analysis of neuropathogenesis of neurocognitive impairment in HIV, Journal of NeuroVirology, 22(4), 431, 10.1007/s13365-015-0410-7
- 9.Paul Robert, Garcia-Egan Paola, Bolzenius Jacob, Mannarino Julie, 2020, , , 50(), 245, 10.1007/7854_2020_185
- 10.Stoff David M., Goodkin Karl, Jeste Dilip, Marquine Maria, 2017, Redefining Aging in HIV Infection Using Phenotypes, Current HIV/AIDS Reports, 14(5), 184, 10.1007/s11904-017-0364-x
- 11.Saylor Deanna, 2018, Neurologic Complications of Human Immunodeficiency Virus Infection, CONTINUUM: Lifelong Learning in Neurology, 24(5), 1397, 10.1212/CON.0000000000000647
- 12.Spagnolo-Allende Antonio, Gutierrez Jose, 2021, Role of Brain Arterial Remodeling in HIV-Associated Cerebrovascular Outcomes, Frontiers in Neurology, 12(), 10.3389/fneur.2021.593605
- 13.Gomez Daniela, Power Christopher, Gill M. John, Koenig Noshin, Vega Roberto, et al, 2019, Empiric neurocognitive performance profile discovery and interpretation in HIV infection, Journal of NeuroVirology, 25(1), 72, 10.1007/s13365-018-0685-6
- 14.Pasipanodya Elizabeth C, Montoya Jessica L, Campbell Laura M, Hussain Mariam A, Saloner Rowan, et al, 2021, Metabolic Risk Factors as Differential Predictors of Profiles of Neurocognitive Impairment Among Older HIV+ and HIV− Adults: An Observational Study, Archives of Clinical Neuropsychology, 36(2), 151, 10.1093/arclin/acz040
- 15.McLaurin Kristen A., Booze Rosemarie M., Mactutus Charles F., 2018, Evolution of the HIV-1 transgenic rat: utility in assessing the progression of HIV-1-associated neurocognitive disorders, Journal of NeuroVirology, 24(2), 229, 10.1007/s13365-017-0544-x
- 16.Paul Robert H., Cho Kyu, Belden Andrew, Carrico Adam W., Martin Eileen, et al, 2022, Cognitive Phenotypes of HIV Defined Using a Novel Data-driven Approach, Journal of Neuroimmune Pharmacology, 17(3-4), 515, 10.1007/s11481-021-10045-0
- 17.Cassimjee Nafisa, Motswai Phenyo K, 2017, Neuropsychological profiles of adults and older adults with HIV, South African Journal of Psychology, 47(1), 35, 10.1177/0081246316646296
- 18.Molsberry Samantha A., Cheng Yu, Kingsley Lawrence, Jacobson Lisa, Levine Andrew J., et al, 2018, Neuropsychological phenotypes among men with and without HIV disease in the multicenter AIDS cohort study, AIDS, 32(12), 1679, 10.1097/QAD.0000000000001865
- 19.Putatunda Raj, Zhang Yonggang, Li Fang, Fagan Philip Regis, Zhao Huaqing, et al, 2019, Sex-specific neurogenic deficits and neurocognitive disorders in middle-aged HIV-1 Tg26 transgenic mice, Brain, Behavior, and Immunity, 80(), 488, 10.1016/j.bbi.2019.04.029
- 20.Brown Monique J., Cohen Steven A., DeShazo Jonathan P., 2020, Psychopathology and HIV diagnosis among older adults in the United States: disparities by age, sex, and race/ethnicity, Aging & Mental Health, 24(10), 1746, 10.1080/13607863.2019.1636201
- 21.Gupta Salil, Venugopal Nirmala, 2020, Risk Factors of Asymptomatic Neurocognitive Impairment in People Living with HIV in an Indian Cohort, Journal of Neurosciences in Rural Practice, 11(), 230, 10.1055/s-0040-1702799
- 22.Shrestha Roman, Karki Pramila, Copenhaver Michael, 2020, , , (), 109, 10.1002/9781119129530.ch6
- 23.McLaurin Kristen A., Booze Rosemarie M., Mactutus Charles F., Fairchild Amanda J., 2017, Sex Matters: Robust Sex Differences in Signal Detection in the HIV-1 Transgenic Rat, Frontiers in Behavioral Neuroscience, 11(), 10.3389/fnbeh.2017.00212
- 24.Cody Shameka L., Vance David E., 2016, The neurobiology of HIV and its impact on cognitive reserve: A review of cognitive interventions for an aging population, Neurobiology of Disease, 92(), 144, 10.1016/j.nbd.2016.01.011
- 25.Krueger Kristin R., Adeyemi Oluwatoyin, Leurgans Sue, Shah Raj C., Jimenez Antonio D., et al, 2017, Association of cognitive activity and neurocognitive function in blacks and whites with HIV, AIDS, 31(3), 437, 10.1097/QAD.0000000000001316
- 26.Ng Rachel QM, Yip KF, Teh YE, 2023, An overview of neurocognitive impairment in older people living with HIV, Proceedings of Singapore Healthcare, 32(), 201010582311606, 10.1177/20101058231160605
- 27.Fazeli Pariya L., Cheatwood John D., Hopkins Cierra, Vance David E., Shirey Maria R., et al, 2023, Association between personality and cognitive performance in middle-aged and older adults with human immunodeficiency virus (HIV), Applied Neuropsychology: Adult, 30(3), 279, 10.1080/23279095.2021.1935954
- 28.Bissel Stephanie J., Gurnsey Kate, Jedema Hank P., Smith Nicholas F., Wang Guoji, et al, 2018, Aged Chinese-origin rhesus macaques infected with SIV develop marked viremia in absence of clinical disease, inflammation or cognitive impairment, Retrovirology, 15(1), 10.1186/s12977-018-0400-y
- 29.Yarandi Shadan S., Duggan Michael R., Sariyer Ilker K., 2021, Emerging Role of Nef in the Development of HIV Associated Neurological Disorders, Journal of Neuroimmune Pharmacology, 16(2), 238, 10.1007/s11481-020-09964-1
- 30.Paul Robert, 2019, Neurocognitive Phenotyping of HIV in the Era of Antiretroviral Therapy, Current HIV/AIDS Reports, 16(3), 230, 10.1007/s11904-019-00426-9