Abstract

INTRODUCTION

Insomnia is a very prevalent and significant sleep disorder associated with reduced quality of life, increased healthcare costs, and increased risks for serious psychiatric and medical comorbidities.^2,3 Over the past four decades, a bourgeoning body of research has focused on the epidemiology, causes, consequences, and treatment of this condition. This research has advanced our understanding of the manifestations and management of insomnia. Yet, as noted in a previous consensus report,¹ the collective impact of these studies has been limited by a lack of standardization in insomnia research methodologies. This lack of standardization, in turn, has contributed to inconsistencies in study findings that have hindered our ability to translate research findings into clinical practice.

This lack of standardization spilled over to some of the most basic tools used in insomnia research, such as the sleep diary. While there is widespread agreement that a sleep diary should routinely be included in insomnia research,¹ the absence of a standardized and widely used sleep diary has compromised the ability to fully interpret and integrate results of previous studies.⁴ For example, meta-analyses of insomnia treatment studies have primarily relied on sleep diary data to estimate treatment effect sizes.^5–8 However, these effect size estimates were based on data from a variety of sleep diaries with distinctive instructional sets and inconsistent definitions of target sleep measures.

Researchers agree that having insomnia sufferers prospectively self-monitor or record their sleep on a night-by-night basis with a sleep diary is a useful methodology for assessment and for tracking treatment effects.^1,9,10 Moreover, there is agreement that such self-monitoring should yield information about a number of relevant metrics including nightly sleep onset latency (SOL), wakefulness after initial sleep onset (WASO), total sleep time (TST), total time spent in bed (TIB), sleep efficiency (SE, i.e., the percent of the time asleep out of amount of time spent in bed), and sleep quality or satisfaction, which reflects a subjective global appraisal of each night's sleep. On the other hand, researchers have not agreed on the format of the sleep diary. In fact, multiple lab-specific sleep diaries have emerged, with response formats including numerical sleep/wake estimates, Likert ratings, and visual analogue scales. Diaries also vary as to whether respondents were asked to provide estimates of all of the key parameters or whether some parameters, such as TST and WASO, were calculated from other parameters. Finally, researchers have used various definitions and different calculations for indices such as SE and WASO (including or not including the final awakening), the primary measures of interest in many insomnia clinical trials.

Despite the lack of a standardized format, the sleep diary has been regarded as the “gold standard” for subjective sleep assessment. Clearly the adoption of a standard sleep diary would be a major step toward moving the field forward. Based on these considerations, the development of a standardized, consensually supported sleep diary is long past due.¹

The development of such a diary, however, faces a number of challenges. The considerable diversity among the sleep diaries used in previous research suggests that there are many different points of view about: (1) the range of questions that should be included and how these questions should be worded; (2) how the diary should be formatted; (3) whether the sleep diary should elicit quantitative responses, qualitative responses, or both; (4) how common sleep parameters (SOL, WASO, SE) should be defined; (5) how much data should be acquired on a single sleep diary form (one day, one week, two weeks, etc.); and (6) what time of day respondents should complete diary questions (i.e., in the morning, in the evening or both). Reaching consensus among insomnia researchers would be an important first step.

Qualitative research on the acceptability of the diary by patients and research subjects would also be important to determine and has not been carried out for existing diary versions (for example, whether they understand the diary questions as intended and if they able to complete the diary easily over extended self-monitoring periods without incurring undue burden). Indeed, the U.S. Food and Drug Administration has suggested that soliciting patient-reported outcomes (PROs) is an important piece in the development and validation of measures to be used in labeling studies.¹¹ Thus, a consensus-based standardized sleep diary that is also informed by patient/user opinion could greatly advance the field.

Recognizing the challenges these considerations pose, a workgroup was formed to develop and propose a consensus-based standardized sleep diary. The specific aims of this work-group were to: (1) review the range and nature of sleep diaries previously used in insomnia research; (2) solicit diaries currently used from a large group of insomnia researchers; (3) identify commonalities in previously used sleep diaries and integrate those commonalities into a consensus instrument; (4) solicit critical input about the resultant consensus sleep diary from a wide range of insomnia researchers; (5) conduct qualitative field testing with patients to acquire additional information to be used for refining the diary instrument and its associated instructional set; and (6) conduct preliminary lexical analyses on the reading level required for the measure.

METHODS

The consensus sleep diary project was an outgrowth of the 2005 Insomnia Assessment Conference which resulted in a publication with recommendations for standard research assessment of insomnia.¹ The original conference was convened in Pittsburgh, PA, and was comprised of an Organizing Committee (consisting of Daniel J. Buysse, Sonia Ancoli-Israel, Jack D. Edinger, Kenneth L. Lichstein, and Charles M. Morin) and 20 invited insomnia experts selected for their research contributions to the field of insomnia as well as for their broad representation of different types of insomnia research.

Beginning in 2008, the five members of the organizing committee and two additional insomnia researchers (Colleen E. Carney and Andrew D. Krystal) participated in a workgroup convened to develop a consensus sleep diary. This workgroup held a number of conference calls for planning purposes and then convened for a face-to face meeting in Chicago, IL, on April 28-29, 2008 to discuss general objectives and strategies for developing a consensus sleep diary. The group agreed on the following decisions concerning the general purpose and format of the diary to be developed. Additionally, the group agreed upon the method whereby items would be evaluated:

• The Consensus Sleep Diary (CSD) would be developed primarily for the purposes of insomnia research. However, the structure of the CSD would also be general enough to be useful for clinical and research applications for all sleep disorders as well as for good sleepers.

• The CSD would build upon sleep diaries previously used in insomnia research to facilitate comparisons across past and future studies.

• Recommendations regarding the CSD from the larger group of insomnia experts who attended the 2005 Pittsburgh Assessment Conference¹ would also be captured. For example, there would be consistency in the calculation of sleep indices described in Buysse et al.¹ and in the new CSD (see Table 1).

• Alternate forms of the CSD would be developed to offer researchers optimal flexibility in addressing research questions. A “Core” CSD would be developed that included a standard set of items minimally sufficient for sleep diary-based research while an expanded form would allow applications that require tracking daytime data (e.g., napping, caffeine use, daytime alertness).

• Previously developed diaries would be solicited from the insomnia research community, including Pittsburgh Assessment Conference members, and these diaries would be analyzed, compared, and discussed among the CSD Workgroup members to identify those items most commonly used and likely to be supported by the insomnia research community.

• The Core CSD would be designed to fit on a single 8.5” × 11” page in landscape format, whereas the expanded CSD version would be designed to fit on the front and back sides of a single page. This was primarily a practical solution that was made in order to both save paper and to appear less daunting to a diary user.

• The wording of the diary items would be in past tense since they are most often completed the morning after the night being reporting upon.

• All questions would be written in the second person so that it would appear as though the researcher/clinician was inquiring about the sleep behaviors.

The workgroup then solicited copies of sleep diaries from all 25 members of the original Pittsburgh Assessment Conference. Items from the submitted diaries were grouped according to item content and placed together on a worksheet for review. For example, items that assessed respondents' estimates of how long it took to initially fall asleep were grouped together. The worksheet had no indication of the original sources.

Once all sets of items were placed on the worksheet, each member of the workgroup rated their top three choices for each item's wording. Members were also permitted to suggest new wording if no choices were desirable. For example, if an item set contained no options worded in the past tense or second person, new items were constructed. The results of the item rating and the rewording process were circulated among members, who were blind to the identity of the person suggesting the reworded items. The workgroup then reviewed the results of the item ratings and rewording via teleconference and agreed upon the final wording of each item. In many cases there was a high degree of agreement. For cases in which the group was divided about the optimal item wording, consensus was achieved through discussion. The workgroup also discussed whether each item would be included as part of the Core CSD or remain as an optional item on the expanded CSD. The results of this teleconference yielded an initial draft of the core and expanded CSD items.

The items were then entered into a self-report tabular format which was discussed on a subsequent teleconference. The workgroup agreed to include guidelines for respondents within the diary itself to decrease the likelihood of common mistakes. For instance, the workgroup agreed it was important to include indicators for whether a time was AM or PM, denoting whether the number refers to minutes or hours, and including tick boxes next to qualitative Likert scale items. A sample column was included to model the desired format of responses. Once the format was agreed upon, instructions for the diary were written. In formulating these instructions, workgroup members discussed their clinical observations of common pitfalls in completing diaries. For example, some patients/respondents have difficulty ascertaining if diary questions pertain to the previous or ensuing night. It was agreed that the instructions needed to be explicit and written at or below an eighth-grade reading level. The first author (CEC) then drafted a set of instructions for the diaries and circulated them for the other members to discuss in a subsequent teleconference on which the instructions were further revised to ensure they were sufficiently clear. The reading level tool in Microsoft Office 2007 Word was used to determine the reading level.

Draft versions of the core and expanded (i.e., optional items) CSD along with their instructional sets were then circulated to the 2005 Pittsburgh Assessment Conference members for feedback. In addition to the CSD and instructions, a questionnaire was also included. The diaries and instructions were re-edited based on the responses. The workgroup then conducted qualitative research with potential respondents' focus groups.

RESULTS

The solicitation of sleep diaries versions from the 25 members of the original Pittsburgh Assessment Conference yielded replies from 22 members (88% response rate). Three members did not respond to multiple contact attempts. Of the 22 sleep diaries that were submitted, only 16 were unique, as some versions were in use across more than one site.

There were seven replies (35%) to the request for clinician feedback on the initial draft of the consensus sleep diary. The first four questions of the questionnaire were Likert scale items (rated from 0 = not at all understandable to 3 = very understandable) that asked: (1) Are the instructions clear? (mean 3; SD = 0); (2) Are the Core items clear? (mean 3; SD = 0); (3) Is the Core diary format clear? (mean 2.8; SD = 0.45); (4) Is the Optional diary format clear? (mean 2.6; SD = 0.55). The next question asked: (5) Should the diary include both evening and morning sections? Of those responding, 50% reported there should be an option for completing some items at night, 33% reported that evening completion should not be an option, and 17% reported that this option should be available for asking about daytime functioning only, e.g. naps, energy, mood. The last question asked: (6) Was there something we missed? Over half (57%) approved of the CSD in its initial format. One respondent requested the addition of an adherence item (e.g., whether the person was following a treatment recommendation such as stimulus control); however, it was decided not to include this item as it was only relevant for treatment trials. Another suggestion was to include direct estimate of TST as a Core rather than optional item; however the decision was made to derive this variable from other information in the diary, as recommended by Buysse et al.¹

Three versions of the final consensus sleep diary were created and are shown in Figures 1​1–3.

The Core Consensus Sleep Diary: The CSD (Figure 1) contained 9 items considered by the CSD workgroup and the 2005 conference participants to represent the most critical parameters. The questions ask about: (1) the time of getting into bed; (2) the time at which the individual attempted to fall asleep; (3) sleep onset latency; (4) number of awakenings; (5) duration of awakenings; (6) time of final awakening; (7) final rise time; (8) perceived sleep quality (rated via Likert scale); and (9) an additional space for open-ended comments from the respondent. As previously agreed upon, the core CSD was formatted so that one week of nightly sleep data could be recorded on a single diary page. The CSD instructions included general information, such as what to do if the respondent misses recording on a particular day, and item-specific instructions to enhance likelihood of correct item interpretation. For example, the instructions for item #6 tell the respondent to record the time of the final awakening in the morning. The additional instructions indicate that all of the items are to be completed in the morning within one hour of getting out of bed.

The Expanded Consensus Sleep Diary for Morning:An expanded version of the CSD (Figure 2) included a number of optional items that could be completed in the morning upon arising (Optional morning completion items – CSD-M). The CSD-M includes additional items about early (premature) morning awakenings (EMA), estimated total sleep time, Likert scale rating of the refreshing quality of sleep, napping/dozing, and alcohol, caffeine, and medication use. The instructions for the additional questions in CSD-M also stipulate that the diary should be completed in the morning.

The Expanded Consensus Sleep Diary for Evening: A third version of the CSD included the same items as CSD-M, but with instructions for morning and evening completion (Optional morning and evening completion – CSD-E; Figure 3). The morning and evening items in the CSD-E are grouped separately. The instructions stipulated that items about daytime activity such as caffeine, alcohol, and medication use or napping which appear on one side of the diary are to be completed at night before going to bed, while the remaining items which appear on the other side and query about the previous night's sleep are to be completed the following morning.

DISCUSSION

A consensus sleep diary, based on input from a large group of insomnia experts and on focus groups of individuals with and without sleep disorders, was developed. Sleep diaries have been universally used as the preferred method for collecting data over time on self-reported sleep and related function in insomnia research.^1,9,10 However, the lack of standardization in diaries currently used in insomnia research studies has posed a significant impediment to progress in the field.¹ The development of a Consensus Sleep Diary addressed this concern.

Advantages of the CSD in particular include the use of both expert consensus and qualitative patient input, as recommended for development of reliable and valid patient-report instruments.²⁰ This methodology included collecting and incorporating diaries currently in use in the field, consensus group editing and composing of items, employing focus groups to solicit input of subjects/patients both for diary development and for editing, solicitation of feedback from experts in the insomnia research community, and carrying out lexile analysis.

The process of developing the CSD also provided new information about sleep diaries and about how people conceptualize their sleep. Among the leading insomnia experts in the field consulted, 16 unique sleep diaries were being used and these varied in terms of the items included, in the item formats used and in their layout. After considering these and other possibilities it was possible to reach consensus on core diary items, their format, and layout as manifested in the final version of the CSD. Beyond the consensus core items, a variety of optional items were proposed, reflecting the varying aims and constraints of research studies, the different contexts in which the diary was anticipated to see application, and the specific preferences of individual investigators. Further, subjects/patients were in agreement on the quantitative dimensions of their sleep that they considered to be important, and this was reflected in the items included in the final CSD. However, participants/patients also felt that the diary format limited their ability to adequately convey the entirety of their sleep experience, and they advocated for additional means to describe these experiences. Thus, the CSD and other diaries should be seen as providing information about some, but not all, patient-relevant aspects of sleep. This theme could be seen as a commentary on interactions regarding sleep with health care providers in general, beyond what a sleep diary is able to capture. Sleep diaries must be augmented with other tools such as other patient-reported outcome instruments and clinical tools where a broader assessment of sleep is needed. The focus group data also suggested that many individuals prefer alternative formats for sleep data representation (clock faces, time lines). Although the consensus of the group dictated against employing these formats in the core diary, such alternative formats could be considered for specific applications.

In summary, focus group participants clearly indicated agreement between the quantitative dimensions of sleep they considered to be important and those included in the diary. However, participants also advocated for a more nuanced and experiential description of sleep. In clinical and research settings, these more qualitative aspects of sleep and wakefulness may be captured by other more efficient methods such as specific patient-reported outcomes or clinical interviews. Finally, multiple participants advocated a more “visual” approach to quantification of sleep (clock faces, time lines) and requested simpler formats. Developing alternate methods for administration of the CSD (e.g., electronic versions, single-day formats) may be useful to match the preferences of specific respondents.

The CSD presented in this document represents a critical first step in the process of standardizing a sleep diary for use across a wide-range of research and clinical applications. It is important that additional work investigate the reliability and validity of this instrument. There are some inherent challenges in establishing reliability and validity with conventional psychometric analyses. For example, one way of assessing reliability is to carry out diary assessments in the same individuals at two or more time points; this establishes test-retest reliability. However, because sleep is known to be highly variable across nights, it may violate the constancy (i.e., temporal stability) assumption of test-retest reliability.²¹ A valid assessment of test-retest reliability for a sleep diary would likely require many nights to overcome the problem of instability in sleep itself.²² Other reliability indices assess the internal consistency of the measure; however, the sleep diary is not intended to measure one construct, and items are not expected to correlate with one another (e.g., there is no reason to expect that bed time would correlate with number of awakenings). One possibility is to focus on the validity of the diary. Construct validity was explored in the current project via user/focus group feedback and expert feedback (both within and outside the Consensus workgroup). It would be helpful to have more exploration of the construct validity of the CSD. For example, future studies could compare the diary data across groups in which the diagnosis has been established according to clinical diagnostic criteria (e.g., healthy controls, insomnia patients, other groups of interest). One could also establish the convergent validity of the CSD by comparing CSD data with data derived from other means of assessing sleep such as polysomnography (PSG), actigraphy, and other self-report instruments. However, the prospective subjective perception of sleep in a sleep diary is a unique and valued perspective, and thus, might not necessarily be expected to agree with objective measures (e.g., PSG, actigraphy). Despite the fact that sleep diaries are widely used in clinical and research settings, with few exceptions,²³ almost no formal reliability and validity data have been presented. One of the reasons for the paucity of data may relate to the unique challenges mentioned above. Although the CSD is unique in that it is the first diary to be developed with rigorous methodology for patient-reported outcome development, devising creative ways to evaluate the reliability and validity appropriately will be an important next step towards the goal of standardizing sleep diary assessment in insomnia research and practice.

DISCLOSURE STATEMENT

The 2005 Pittsburgh conference and manuscript preparation were supported by an educational grant from the National Institute of Mental Health; by unrestricted educational grants from Pfizer, Inc; Sanofi-Aventis, Sepracor, Inc; Takeda Pharmaceuticals; and by The Mental Health Intervention Research Center at the University of Pittsburgh (MH30915). Industry sponsors did not contribute to CSD conference planning, nor did they participate in any aspect of manuscript preparation. Dr. Carney has no conflicts of interest to report. Dr. Buysse has served as a consultant (for less than $10,000 in fees) for the following companies: Actelion, Cephalon, Eisai, Eli Lilly, GlaxoSmithKline, Merck, Neurocrine, Neurogen, Pfizer, Philips, Purdue Pharma, L.P., Sanofi-Aventis, Sepracor/Sunovion, Somnus Therapeutics, Takeda, and Transcept Pharmaceuticals Inc. Dr. Buysse has helped produce CME materials and lectures indirectly supported by Sanofi-Aventis, Sepracor/Sunovion and Takeda, and has been paid for lectures at non-CME educational meetings by Servier. Dr. Ancoli-Israel has served on the following Consultant/Scientific Advisory Boards (for less than $10,000 in fees): Ferring Pharmaceuticals Inc., GlaxoSmithKline, Johnson – Johnson, Merck, NeuroVigil, Inc., Pfizer, Philips, Purdue Pharma LP, Sanofi-Aventis, and Somaxon. Dr. Edinger has research support from Philips/Respironics and has consulted with Live to Sleep, Inc. (for less than $10,000 in fees). Dr. Krystal has received grants/research support from: Sanofi-Aventis, Cephalon, GlaxoSmithKline, Merck, Neurocrine, Pfizer, Sepracor/ Sunovion, Somaxon, Takeda, Transcept, Respironics, Neurogen, Evotec, Astellas, Abbott; and has served as a consultant for: Abbott, Actelion, Arena, Astellas, Axiom, AstraZeneca, BMS, Cephalon, Eli Lilly, GlaxoSmithKline, Jazz, Johnson and Johnson, King, Merck, Neurocrine, Neurogen, Novartis, Organon, Ortho-McNeil- Janssen, Pfizer, Respironics, Roche, Sanofi-Aventis, Sepracor, Somaxon, Takeda, Transcept, Astellas, Kingsdown Inc. Dr. Lichstein has served on an advisory board for Merck and has received research support from Take-da. Dr. Morin has received research support from: Merck and Sanofi-Aventis, and has served a consulting/advisory role for Merck, Sanofi-Aventis, and Eli-Lilly.

ACKNOWLEDGMENTS

Footnotes

REFERENCES