The authors have declared that no competing interests exist.
Mental health promotion depends to a large extent on the gathering of accurate baseline opinion data from a targeted population.
The aim of this paper is to explore the potential applications of a new method of measuring true attitudes toward mental illness, and of monitoring and evaluating subsequent public health interventions.
This paper reviews the strengths and limitations of a novel survey method, RIWI, its early findings, and its potential applications in the field of mental health promotion. Comparisons are made to other commonly used survey methods through entering pertinent search terms into the Google Scholar™ database.
The RIWI online survey method has several advantages over earlier survey methods: it is random, quick, anonymous, and reaches very large samples. Questionnaires are easily translatable and can be repeated, with excellent test-retest reliability. Anonymity reduces social desirability bias. The limitations are: a) variable completion rate, which carries the upside of allowing regional comparisons, and b) the reality that the respondent pool reflects regional Internet usage, often biased toward young literate males.
A survey method that is able to quickly and repeatedly sample large numbers of random individuals is an important advance for health promotion in that interventions can be timely and their efficacy can be rapidly evaluated.
Before launching initiatives in mental health promotion, it is important to accurately survey the views and beliefs of local stakeholders. Reliable surveys enable what is referred to as ‘ground-truthing,’ candid opinion provided by voluntary self-report as opposed to information gathered through indirect inference. The aim of this paper is to explore the potential applications of a new survey method to measure attitudes toward various aspects of mental illness and, thus, to enable subsequent testing of the adequacy of public health interventions.
This paper first describes a novel Web survey method, Random Domain Intercept Technology (RDIT™), addresses its strengths and its limitations, and outlines early findings based on the method. Subsequently, the authors systematically entered the search terms defined in
Population Breadth | The ability to reach as many diverse individuals as possible |
Size of Sample | The ability to collect large population samples |
Randomness | The equal chance of any individual being exposed to the survey questions |
Representativeness | The degree to which respondents represent the general population of the region |
Anonymity | The impossibility of associating respondents with their answers, thus guaranteeing confidentiality |
Freshness | The likelihood of recruiting respondents with minimal previous exposure to surveys |
Voluntariness | The certainty that potential respondents have the freedom to participate or not, as they wish |
Appropriateness of Questions | The likelihood that the language, tone, and clarity of the questions enables respondents to answer accurately |
Appropriate Length of Questionnaire | The likelihood that the questionnaire is long enough to cover the research question but short enough to prevent survey fatigue and that it thereby ensures sufficient numbers of reliable and complete responses |
Validity of Questionnaire | The likelihood that the questions asked will yield quantitative or qualitative answers useful to the research purpose |
Neutrality | The likelihood that respondents will not be swayed by what they consider to be socially desirable answers or by the offer of rewards for participation |
Reproducibility | The likelihood that the same opinion questions, administered more than once to the same population parameter, will yield the same results (unless there is an intervening event that explains the discrepancy in results) |
Follow-Up | The possibility of administering a questionnaire repeatedly, thus enabling longitudinal research and the measurement of efficacy of an intervention |
Taking advantage of the fact that all users of Web-enabled devices make occasional typographical errors on non-trademarked websites when navigating the Web, errors that lead them to unintended Internet destinations, the RIWI method places survey questions on such unintended Web destinations
There are several known methods of gathering opinion data about mental health and illness. Each one has its strengths and biases. Face-to-face or telephone interviews can collect in-depth information. A trained interviewer is often able to motivate respondents to respond fully and honestly by building a relationship of trust, which is absent in impersonal Web surveys. On the other hand, such interviews are not anonymous so that candid responses are constrained by social desirability bias and impression management, the tendency of survey respondents to answer questions in ways that interviewers would approve of
Person-to-person interviews have the advantage of being able to use long, standardized, reliable, and valid questionnaires, which are able to yield, for instance, accurate point prevalences of mental health conditions
In contrast to other methods, the RIWI process can reach vast numbers of individuals simultaneously throughout the world, and its brief questions can be easily translated into any number of languages, as needed.
Delphi–type panels, structured communication methods that rely on the judgment of a panel of experts, have been used to gather a wide range of mental health data
Another way of extracting qualitative information about mental health is from applying linguistic algorithms to the text of postings on Twitter™ or other social media and blogs. The method has been applied to the analysis of public sentiment about autistic spectrum disorder
A problem with all Web-based surveys is that the relative intensity of Web usage varies among regions. Offline surveys have similar limitations in that some populations are more accessible than others. The problem is partially mitigated by the RIWI process via respondent population re-weighting to local Census data and through the participation of large samples of all age categories.
Most factors important to consider when designing mental health surveys intended to cover as wide a swath of the world population as possible are summarized in
The meaning of “response rate” differs in different contexts, which makes survey methods difficult to compare on this measure. With respect to the RIWI method, the sample potentially exposed to the questionnaire is every person using the Internet in the targeted location during the time of the study. Whom the survey reaches depends on how frequently one uses the Internet, what access one has, whether one prefers to type or to copy and paste URL addresses, whether one tends to use bookmarks, and how careful one is to avoid mangled words or non-existent websites in the URL bar (everyone makes such errors on non-trademarked domains; some of us more frequently and some of us less so). It cannot be known how many persons are presented with the questionnaire and choose to ignore it. The main reason for ignoring the questionnaire is presumably the wish to return to the true, intended website destination. The number of such “non-responders” is unknown. There may also be “non-responders” who briefly scan the questionnaire and decide against it for lack of interest or other reasons, but this number is also unknown. An analogy is any member of the general public who sees a newspaper advertisement for a survey and ignores it. Such people are not included in the denominator when calculating response rates.
For RIWI and for online newspaper advertisements, non-response can be defined as beginning a survey and then abandoning it – i.e., abandonment rate. The post opt-in RIWI ‘abandonment rate’ varies depending on the topic and on the geographies targeted. Abandonment rates are generally lower in wealthier than in poorer countries; they depend on the salience of the topic to respondents
There may be some sensitive items on a questionnaire that are less likely to be responded to than others. This is the “item response rate” and it will vary with the sensitivity of the question and how it is framed. RIWI’s ability to reach non-habitual responders who can be expected to be relatively interested in answering survey questions and RIWI’s receipt of approximately the same number of responses from any given region at different points of time both suggest fairly robust responsiveness, although exact comparisons with other methods are not possible to make.
RIWI respondents currently come from 231 countries and territories and from all walks of life, but the respondent pool is limited to those with access to the Internet (43% of the world’s population) – and, generally, therefore, this means more males (65%). The male/female ratio varies, depending on geography. The male bias is narrower in North America and Western European countries than it is in the rest of the world. For example, the population with access to the Internet is 52% male in the U.S., in contrast to 77% male in Pakistan
The RIWI method does not provide financial incentives to participants and collects no personally identifiable information about individual respondents. The anonymity and lack of incentives dramatically reduce social desirability bias
Because of the randomness of recruitment, RIWI respondents tend to be unjaded, relatively ‘fresh’ with respect to participating in surveys. The GRIT Consumer Participation in Research Report
To check test retest reliability, RIWI has conducted monthly repeats of the same questions in a well-populated country (India) over 21 months
The large sample sizes that can be obtained through the use of this method ensure the statistical power required to obtain significant results for questions posed in a brief survey. In a study for the World Bank, over 16,000 complete responses were captured in a month using RDIT™ methodology, and as many as 60,000 responses to individual questions in 14 countries
RIWI surveys are cross-sectional and cannot follow the same individuals over time. However, large samples of comparable individuals can be recruited and the survey repeated as often as needed. This enables longitudinal research.
All Web surveys, especially if no incentives are provided, tend to have large drop off rates, questionnaires being abandoned before completion. The 40% RIWI completion rate may sound low but there has been a lack of consensus on how best to calculate and report response rates on surveys, understandably, because dissemination methods vary. Surveys traditionally aim at a somewhat arbitrary 60% response rate, but what counts most is sample representativeness and response rates may not be as strongly associated with representativeness as was once believed
It is to be expected that a fairly large proportion of people exposed to RIWI questions will choose not to answer them, but this does not bias the survey results because of the large potential pool of responders and the random nature of the exposure.
All Web surveys select for respondents with Internet access. As time passes, Internet access will be available to more and more of the world’s population and, as smartphone and tablet usage increases and the number of Web domains in each country increases, manual type-in errors on all browsers will become more likely, thus increasing the potential reach of RIWI surveys
The method works best with very brief sets of questions so that lengthy mental health questionnaires are not used. This makes the method challenging for lengthy epidemiological surveys of the prevalence of mental health conditions that require well-validated gold standard instruments. Long RIWI surveys can, however, be modularized, or ‘chunked up’, as has been done in one Canadian study on social values that consisted of a 150-question survey instrument. The results showed good reproducibility of long-established norms
The RIWI method was first used to probe anti-vaccine sentiment during the 2009 HINI flu pandemic
More recently, RIWI conducted a global survey of mental health stigma, probing attitudes toward persons suffering from mental illness
In developing countries, 15% of respondents (twice as many as in developed countries) were of the opinion that individuals with mental illness were violence-prone. The developing/developed country difference in this form of stigma can be attributed to the fact that mental illness is often left untreated in developing countries because of poor access to mental health services and perhaps also to the available treatment being relatively ineffective. According to the World Health Organization, more than 75% of persons with serious mental illness in less-developed countries never receive treatment and for those who do, treatment effectiveness has not been evaluated
While 45% to 51% of respondents from developed countries believed mental illness to be similar in kind to physical illness, only 7% agreed that mental illness could be overcome. This was a large discrepancy, suggesting that biophysiological perspectives on mental illness do not necessarily lead to trust in the effectiveness of treatment.
The findings of this study and its methodology captured significant academic attention
The RIWI method lends itself to health promotion because it can provide: a) quick accurate responses from very large, random, and widespread populations b) anonymous answers to questions that are sensitive in nature c) answers to brief, validated survey instruments. This method can probe regional differences and questions can be asked before and after public health interventions so that intervention efficacy can be measured. Repeating the survey can also test the duration of impact of a public health campaign. Furthermore, the participant abandonment rate can, in the proper context where, for example, an anti-stigma campaign is active, be a good measure of the resonance of the message of that campaign. The lower the percentage of respondents who fail to respond or complete, the higher the evident interest. This intensity measurement is more transparent and objective than sentiment analysis based, for instance, on the application of language algorithms to social media postings that need to be interpreted
The rest of the discussion outlines mental health conditions to which the RIWI survey method can be applied. The examples were chosen by the authors to reflect public health need, requirement for anonymity, availability of appropriate validated survey instruments, presence of regional differences, and the possibility of using survey results to evaluate an intervention.
Prevention of posttraumatic stress syndrome (PTSD) is a particularly good example of a condition to which this kind of survey can be profitably applied because PTSD is a reaction to an event. It is an important syndrome to study because it inflicts substantial burden on individuals and on societies across the globe
Comorbidity of PTSD and substance abuse
Substance abuse on its own brings associated problems that pose an increasing challenge throughout the world although the actual substances of abuse vary from region to region. Alcohol and opiate use is escalating in Europe, Africa and Asia, while amphetamine and cannabis use is rising in Asia, North America and Europe. Cocaine use is on the rise in North America and Europe, whereas khat is a problem in parts of Africa and the Middle East, as are coca leaves in South America
Professional education is important as physicians who prescribe analgesics, hypnotics, anxiolytics, or stimulants can inadvertently contribute to the addiction problem. Medical prescription often starts people on the road to addiction. Surveying jurisdictions before and after continuing medical education campaigns can help to design more effective educational interventions.
There are forms of non-drug dependency (e.g., gambling, sex, shopping, hoarding) that are widely considered to be Western phenomena – their presence or relative absence in the developing world can be determined by surveys. Answers to such questions are important for global disease prevention and health maintenance.
Much psychopathology will be preventable when more is known about negative emotions such as shame and guilt
The RIWI method can respond quickly to new events. For instance, because of events in Iraq and Syria, the world is currently on high alert to the effects inherent in large-scale migrations of refugees. RIWI surveys can elucidate the relationship between refugee status, trauma, shame, and guilt, and determine whether the emotional needs of migrants are addressed more effectively by specific resettlement strategies
Particular segments of the population may require special focus. The elderly, for instance, are more vulnerable than other groups to some mental illnesses. The growth in the prevalence of dementia has been identified as a priority at a recent G-7 Dementia Research Meeting
Related to the prevalence of old age and dementia in the community is the issue of prevention of elder abuse. Though the victims may not have access to the Internet, questions pertaining to elder abuse can be asked of the general population who have observed acts of physical, emotional, sexual, and financial abuse directed toward the frail elderly. Here again, the effects of population educational campaigns and changes in the licensure of care providers can be assessed by repeat surveys.
Survey respondents can be forthright about sensitive topics when the surveys are anonymous. As an example, a recent study found that heterosexual respondents were less likely to support equal access to employment, military service, adoption, and marriage for homosexual people in Web surveys (where they perceived themselves to be anonymous) than when being asked the same questions face-to-face
Social desirability plays a large part in the accuracy of survey responses. The issue of induced abortion, for instance, is one that divides people across the world. Many consider taking a life to be an indefensible crime. Others believe abortion is a woman’s choice since she is the one carrying the fetus. A question that has troubled even the staunchest abortion advocates is sex-dependent abortion, the preferential killing in utero of female fetuses. Only an anonymous, entirely confidential survey can garner people’s true opinions on such sensitive issues, and this is vital to the passage of public health legislation that reflects majority opinion.
A good example of a condition for which many quasi solutions are implemented without prior evaluation is obesity
Attention deficit hyperactivity disorder (ADHD) is an example of a contested diagnosis, given to a child who is inattentive, impulsive, and overly active at home or at school. It has a worldwide prevalence of about 5%
Regional variations can help. While there is a global interest in attention deficit hyperactivity disorder
Concussion associated with sports trauma, especially with respect to late effects, has become a major preoccupation
The RIWI survey method is able to obtain simultaneous answers to brief focused questions on sensitive issues from very large numbers of random respondents. Comparable incentive-based surveys take longer to complete and may be less representative and more vulnerable to bias. Repeat surveys are needed to answer many important health questions and have, until now, been particularly difficult to conduct, especially in developing countries, because of cost and lack of available resources.
A recent paper predicts that the burden of mental health and substance-use disorders will increase worldwide in the near future and that the greatest rise will be seen in low and middle-income countries because of rising life expectancy in those regions, population growth and under-resourced health care systems