Home About us Editorial board Search Ahead of print Current issue Archives Submit article Instructions Subscribe Contacts Login 
  • Users Online: 151
  • Home
  • Print this page
  • Email this page


 
 Table of Contents  
REVIEW ARTICLE
Year : 2017  |  Volume : 3  |  Issue : 4  |  Page : 154-163

Evolution of the digital biomarker ecosystem


1 Eli Lilly Innovation Center, 450 Kendall Street, Cambridge, MA, USA
2 Clinical & Translational Science Institute, Tufts University Medical Center, 800 Washington St, Boston, MA, USA

Date of Web Publication26-Mar-2018

Correspondence Address:
Graham Barry Jones
Tufts University Medical Center, Clinical and Translational Science Institute, 800 Washington St, Boston, MA 02111
USA
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/digm.digm_35_17

Rights and Permissions
  Abstract 


The pursuit of digital biomarkers wherein signal outputs from biosensors are collated to inform health-care decisions continues to evolve at a rapid pace. In the field of neurodegenerative disorders, a goal is to augment subjective patient-reported inputs with patient-independent verifiable data that can be used to recommend interventive measures. For example, in the case of Alzheimer's disease, such tools might preselect patients in the presymptomatic and prodromal phases for definitive positron emission tomographic analysis, allowing accurate staging of disease and providing a reference point for subsequent therapeutic and other measures. Selection of appropriate and meaningful digital biomarkers to pursue, however, requires deep understanding of the disease state and its ecological relationship to the instrumental activities of daily living scale. Similar opportunities and challenges exist in a number of other chronic disease states including Parkinson's, Huntington's, and Duchenne's disease, multiple sclerosis, and cardiovascular disease. This review will highlight progress in device technology, the need for holistic approaches for data inputs, and regulatory pathways for adoption. The review focuses on published work from the period 2012–2017 derived from online searches of the most widely used abstracting portals.

Keywords: Closed loop, devices, digital biomarkers, medicine


How to cite this article:
Wright JM, Regele OB, Kourtis LC, Pszenny SM, Sirkar R, Kovalchick C, Jones GB. Evolution of the digital biomarker ecosystem. Digit Med 2017;3:154-63

How to cite this URL:
Wright JM, Regele OB, Kourtis LC, Pszenny SM, Sirkar R, Kovalchick C, Jones GB. Evolution of the digital biomarker ecosystem. Digit Med [serial online] 2017 [cited 2023 Jun 8];3:154-63. Available from: http://www.digitmedicine.com/text.asp?2017/3/4/154/228663




  Introduction Top


The landscape for digital biomarkers has evolved through a combination of technology push from instrument and device developers as well as end-user pull from patients, advocacy groups, and more recently providers. In the former case, health-related applications embedded in smartphones and wrist-based monitors have become ubiquitous and commoditized. In the latter, there exists a rapidly growing demand for systems and devices that can diagnose, monitor, and assist in the management of chronic diseases most evident in the neurodegenerative arena. The confluence of these drivers has thus become a dynamic environment, and expectations are high that tangible progress will be made toward improved health-care outcomes. Workshops and symposia that bring together scientists, engineers, clinicians, and patient advocates are starting to drive the agenda,[1] scientific journals are emerging,[2],[3] and venture funding is becoming readily available to help translate ideas into clinical practice.[4] The underpinnings and evolution of the digital biomarker industry are presented along with initial clinical observations.


  The Clinical Value of Biomarkers Top


The term biomarker entered the lexicon of clinical medicine in the 1980s [5] and gained momentum with the definition of “a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention” assigned by the NIH working group in 1988. In clinical parlance, the terms “surrogate marker”[6] and surrogate endpoint [7] “a biomarker intended to substitute for a clinical endpoint” are often seen as more relevant. Nonetheless, biomarkers often represent readily accessible data points, which can be useful where definitive assessment is impractical from both time and financial perspectives. The origin of the term digital biomarkers is less precise, but a formal definition as “consumer-generated physiological and behavioral measures collected through connected digital tools” has been advanced.[4] The use of biomarkers (both conventional and digital) in clinical trial design requires that such markers are appropriately validated, and the US Food and Drug Administration (FDA) has established a biomarker qualification program through the Center for Drug Evaluation and Research, and a stakeholder-based approach which seeks to engage academia, industry, government, and consortia has been outlined.[8] This has stimulated progress toward these goals ranging from efforts to establish digital biomarkers for mood disorders [9],[10] to assessment of functional biomarkers derived from implanted and wearable devices in cardiovascular disease.[11]


  Devices Top


The near explosive growth in consumer products which digitally monitor and track health-related parameters has heightened expectations regarding improved health outcomes. In addition to now, ubiquitous examples of app-laden smartphones and wristbands are a plethora of electronic devices, sensors, and monitors that are worn, embedded in clothing, or implanted [Figure 1]. A long-term objective would be for the integrated data from sensors to inform a response with an effector, for example, the administration of a drug and neurotransmitter or analgesic from an implanted reservoir. Such “closed-loop” systems have already been prototyped, trialed, and commercialized in the type I diabetes market,[12] and new systems for the administration of anesthetic agents in response to processed electroencephalogram signals are also in development.[13]
Figure 1: Potential portable, wearable, and implantable components of radio frequency-based hybrid “closed-loop” health maintenance systems

Click here to view


Portable devices

It has been estimated that three quarters of the world population have access to mobile communication networks and that more than one third of global smartphone users have installed some type of medical application which will rise to >50% by 2018.[14] In the USA alone, smartphone use is expected to reach 80% by 2020 and approximately 30% of current users have at least one health-based app in use.[4] However, it is incumbent on developers to keep consumers engaged in the process as it has also been estimated that one quarter of these apps are used only once, and three quarters are not used more than 10 times.[15],[16] The potential impact of mobile devices on lowering health-care costs is a subject of active analysis, one study calculating that in 2017 alone, >€100 billion in health-care costs could potentially be saved in the European Union (composed of €69 billion on disease prevention and €32 billion on disease treatment and monitoring).[14] One of the first notable examples of integration of smartphones with health monitors was the release of the Nike + wireless chip in 2006. Inserted into the base of certain athletic shoes, it allowed users the ability to record elapsed duration of workouts, including distance travelled, pace, and estimated calories burned when paired to the Apple iPod or iPhone 4 platform.[17] Innovative in design and user adoption, it required purchase of the sensor and a subscription-based service to the Nike + online community and specific athletic shoes capable of hosting the sensor chip. Other competitors in this space followed with similar technology, including the Adidas miCoach among others.[18] Development of smartphone technology-based health apps became popularized en masse in 2013 with the launch of the iPhone 5S by Apple which incorporated a step counter, thus integrating all functions into a single device. The system, comprising an accelerometer, digital compass, and Medication Event Monitoring System (MEMS) gyroscope, allowed users to track activity, set targets, and record data over extended periods. Given the widespread popularity of the phone, its subsequent iterations, and competitor products with similar capabilities, the smartphone has become the platform for myriad additional health-related applications. Based on the trend for increased screen size, durability of the units, and touchscreen alphanumeric keyboard entries, a number of studies have focused on the phone-user interface in disease assessment and management (vide infra). In addition, notable technological advances have emerged, including the development of dedicated smartphone-based mobile health coaches, exemplified by the Ginger.io system.[19]

The use of a commercial smartphone to detect change in heart rate has been stimulated by the success of wrist-based monitors (vide infra) which use photoplethysmography (PPG) to assess cardiovascular outputs. Reflection-based PPG techniques are employed using the smartphone light and camera functions, confirming its versatility as a sensing device.[20] Demonstrating smartphone capabilities in the field of diagnostics, a team recently reported use of a powerful neural network to analyze pixel-based images derived from the device to assess dermatologic cancers.[21] The results were validated independently by board-certified dermatologists, suggesting potential applications in field diagnosis. Very recently, precision analytical devices which couple with commercial smartphones have been reported. In one example, developed by Oxford Nanopore Technologies, a smartphone powered nanopore DNA sequencer was demonstrated.[22] In another case, mobile phone-based microscopy was applied to enable in situ point mutation assays and DNA sequencing reactions.[23]

Handheld device use in health care is not restricted to smartphones. A team has successfully used a composite application on the Apple iPad for at-home assessment of Alzheimer's disease patients.[24] The suite (denoted “C3-PAD”) was used by older individuals in the absence of a psychometrician with encouraging results. In a broadly similar vein, Akili Interactive Laboratories have developed a gaming technology platform able to detect statistically significant differences between participants with and without brain amyloidosis, the primary biomarker for Alzheimer's risk. The study which was conducted in collaboration with Pfizer has presented positive topline data.[25]

Wearable devices

A survey of consumers who have purchased wearable health monitoring devices [17] reported that 45% owned one of a large variety of fitness bands on the market, 27% a smartwatch, 15% smart glasses, 14% a smart video or photo device, and 12% a form of smart clothing. Significantly, respondents indicated that interest in the products waned over time for multiple reasons including re-evaluation of use-case need, ease of misplacing or losing the device, lack of esthetic appeal, poor comfort level, short battery life, and issues surrounding smartphone synchronization. Given the large number of wrist-based monitors in the marketplace, the report is likely to stimulate additional modifications to those devices, which clearly have considerable potential to capture digital biomarkers.[26] An advantage for a wrist-based device (smart band or watch) over a smartphone is the intimate surface contact with the skin of the individual. Furthermore, with appropriate attention to esthetics and comfort, the device could be realistically worn 24 h/day, providing data which is clinically more relevant than serial/batch analysis. The principle biomarkers studied to date have been cardiovascular, capitalizing on PPG. Initial work in this field employed transmission PPG, where infrared (IR) light penetrates skin and underlying tissue, and changes in blood flow in the arteries and arterioles during the cardiac cycle (systolic phase) impact the light path, whose changes are correlated with blood pressure.[27] Sites for measurement include the earlobe, fingers, and more recently wrist. One of the most commonly used PPG sensors is ring based, attached to the finger base. Recent developments in optics technology have resulted in the development of green light-emitting diode (LED) for use in PPG. Green LEDs operating at ~550 nm (as opposed to ~800 nm for IR) have more pronounced interactions with both oxyhemoglobin and deoxyhemoglobin, allowing more accurate pulse oximetry readings, as they are now classified.[27] The technology has been rapidly incorporated into wrist-based devices and will likely feature in next-generation smartwatches. One caveat with the use of green light for PPG analysis is its inability to penetrate >1 mm (the IR band by comparison can penetrate 2.5 mm or greater). Accordingly, reflectance mode PPG is typically employed where the sensor and light source are in proximity. In either mode, it is expected that the signal is derived from both systolic pressure changes in the artery and the impact of those forces on surrounding connective tissues.[27] It is reasonable to assume that arterial plasticity/rigidity and composition of connective tissue impact measurements, presenting both challenges and opportunities for use in chronic cardiovascular conditions and in gerontology studies where specific (trained) algorithms may be needed.[28] Another issue to be resolved is the impact of skin pigmentation on the accuracy of data obtained and remains topic of active debate.[29],[30]

A number of studies have reported that pressure imposed on wrist-based sensors has a beneficial impact on the PPG signal. These externally applied pressures are assumed to impact interstitial and arterial microenvironments and when optimized have a marked impact on the amplitude of the AC signal from the PPG. However, uniform standards for contact pressure have not been established to date. A study of healthy volunteers using applied pressures ranging from 0 to 200 mmHg was conducted and concluded that the highest signal amplitude was obtained at pressure of 60 mmHg.[31] PPG technology has recently been extended to the quantification of respiratory-based variations to the measured PPG waveform, partially allowing the gauging of volume responsiveness.[32] Applications of smartwatches in the management of Parkinson's disease (PD) have been reported, where output data were integrated to a customized smartphone.[33] A key factor in the study lies in that the device is worn continually throughout, providing constant data points. There is major interest in the potential of the smartwatch recently launched by the Verily Corporation.[34] The device, referred to as “Study Watch,” is capable of collecting participants' electrocardiograms (ECGs), heart rate, dermal electrical conductance, and physical movements. It is also able to record light and sound and encrypts all data, which is periodically uploaded to the cloud.[35] Although not available for consumer purchase, it will be deployed in the Personalized Parkinson's Project in the Netherlands and a longitudinal study known as “baseline” which will track 10–20,000 people over a multiyear period in an effort to identify clinically relevant biomarkers. Given that the same company also developed the glucose-sensing contact lens, the potential for multiple point data integration (for example, through a smartphone) may exist.[36] Of relevance to consumer concerns regarding smartwatch battery life (and potentially esthetics), the design of polymer-based rechargeable devices for possible use in wearable electronics was recently reported.[37] The materials are robust and may allow other desirable properties to be incorporated for the use in consumer products. Finally, progress in the miniaturization of sensor technologies could expand the repertoire of biomarkers studied using wearables. One of the most promising lies in Fabry-Perot interferometry, which is tunable from the ultraviolet through IR spectrum. Using optical MEMS approaches, these systems have been miniaturized by the VTT Technical Research Center in Finland, and numerous applications can be anticipated.[38]

Accelerometers

Although accelerometers are now commonly embedded in integrated smart devices, there exist a number of scenarios where a stand-alone monitor has utility in the capture of digital biomarkers. A recent study sets out to measure motor impairment in individuals with Huntington's disease using multiple strategically positioned accelerometer-based sensors.[39] The pilot study, which used five sensors to measure patient gait, showed variation between in-clinic and at-home settings and between patients with higher levels of motor function.[39]

Ocular devices

Much excitement was generated with the announcement of the glucose-sensing contact lens outlined by Verily. The device (often referred to as the Google contact lens) is envisioned as a component of a hybrid closed-loop system, where radio frequency (Rf)-based communication between the biomarker sensor (lens) and effector pump (logically, insulin) allows homeostatic management. The use of contact lens sensors for digital biomarker capture is likely to mature rapidly as it offers many opportunities for insight to health and disease.[40] Tears are a potentially very rich source of relevant biomarkers,[41] and potential diagnostic biomarkers for Alzheimer's disease have been noted.[42] In addition to providing a locus for sensing events, smart contact lenses may also be amenable to enhanced retinal scanning procedures, where prognostic/diagnostic disease markers are monitored at periodic intervals. High-resolution retinal imaging has been applied to identification of preclinical Alzheimer's disease,[43] and it is possible that a combination of imaging and tear sensing could evolve into a highly effective strategy.

Implantable devices

Substantial research has been undertaken in the long-term deployment of biosensors in implanted devices as they represent a critical component of closed-loop insulin systems, with potential applications in myriad indications.[44] One of the principle obstacles to overcome is the foreign body response,[45] which currently limits sensor deployment to a matter of days due to formation of fibrous tissue and inflammatory responses.[46] Advances in device and material design may provide solutions, including the use of polymer coatings.[47] Dental implants offer a potential locus for biomarker sensing in that they enjoy close contact with biologic fluids (saliva and blood), have a permanent base, and could in theory be modified to include appropriate circuitry. The use of such technology for the detection of cardiovascular events has been demonstrated [48] and other applications could be envisioned. In addition, development and strategic deployment of skin-based and implanted nanosensors (e.g., nanotattoos) may have potential for the sensing of various analytes.[49],[50] Flexible devices containing sensors attached to the skin surface are being developed for the analysis of biomarkers [51] including the BioStampRC ® system developed by MC10 Incorporated. In one example, analytes within sweat are channeled through microfluidic arrays then subjected to colorimetric analysis, imaged using a smartphone.[52] Successful integration within closed-loop Rf-based systems for drug delivery could be profound [Figure 1], for example, triggering release of neurotransmitters for patients when low serotonin levels are detected, release of anticoagulants when stroke forming micro clots are detected, or release of anti-infectives when the immune system is triggered. This will require establishment of appropriate biomarkers and technology for quantitation, but guidelines are being advocated [50] and progress made in several key areas.[11]

Ingestible sensors

The ability to track medication dosage is of key significance in certain diseases where noncompliance with regimen can have serious consequence. One method to accomplish this has been demonstrated in the form of ingestible sensors, which are released in the gastrointestinal tract when co-administered with medications. Data are transmitted from the sensor to a receiver placed on the patient torso giving real-time data on drug movement in addition to other biomarkers such as heart rate and activity.[53] These studies have evolved to comprise what are referred to as MEMS and have been demonstrated clinically in central nervous system (CNS) disorders.[54] Coupling these data to other digital biomarkers – for example, those captured from a smartwatch could allow close monitoring of certain neurodegenerative diseases.

While the systems described above offer considerable potential as stand-alone devices, integration of data from multiple sensors and devices to permit clinical grade decision-making is likely to remain a formidable challenge for the foreseeable future. This said that compelling evidence is becoming available in specific therapeutic areas,[55] and given the potential ramifications in disease detection and management, it seems inevitable that these issues will ultimately be addressed.


  Regulatory Pathways Top


The regulatory pathway for approval of digital biomarker-based systems involves a potentially complex landscape, the first of which involves definition on the validity of the biomarker itself.[8] Given progress in the field of precision medicine, there is considerable interest in the development of in vitro companion diagnostics in drug development and the FDA has issued formal guidance.[56] The regulation of wearable devices, however, appears fluid and is described in guidance on mHealth applications.[57] Notable is that the FDA has determined that many health monitors and products designed to promote general wellness are exempt from regulation, a stance which is supported by the device industry.[57] Such is a relief as it has been reported that in just one of these device categories (wrist-worn pedometers), considerable variation in the actual number of steps counted was observed between three branded devices (Fitbit Flex, Nike FuelBand, and Jawbone UP24) which presumably needs to be factored into any comparison groups used in clinical settings.[4] The regulatory landscape is likely to evolve as progress is made in the field and influenced by instrument developers, those engaged in clinical trials, and patient advocacy groups. For example, the International Parkinson and Movement Disorders Society Task Force on Technology is currently making the case for open standards to be introduced which will allow crosstalk connectivity between devices, opening dialog with regulators in the FDA and European Medicines Agency.[15] Similarly, there is evident need for standards which address the interoperability of devices across different platforms. For example, in the case of closed-loop continuous glucose monitors, some systems communicate with the base unit using the 2.4 GHz band (and the IEEE 802.15.4 standard) whereas others use either 868 or 916 MHz channels.[58] The adoption of the 802.15.4 standard is strategic as it avoids the potential for interaction with the most common Wi-Fi channels which are based on the now ubiquitous 802.11 standard. How such sensors can communicate with various smartphone platforms,[59] and the adoption of necessary data security protocols will be an area of active and substantial research and debate.


  The Patient-Centric Ecosystem Top


To fully realize the benefits of digital biomarkers in managed health care, holistic approaches to the patient's environment also need to be factored. This is exacerbated in the case of neuromuscular diseases, where in addition to the patient's physical environment, the interactions with caregivers, family members, and social networks play a key role. Broadly considered, factors which impact patient quality of life include health maintenance, physical function, mental function and social networks, and metrics need to be developed for each in order to assess impact on disease. Many of these factors can be derived from digital sources, for example, monitoring Facebook postfrequencies, proficiency in mental tests administered through a smart device etc., and thus integrated into digital outcome assessment. Driving the agenda for these considerations falls outside the scope of any one instrument developer, but various consortia are rising to the challenge. Among the most prominent is the Coalition Against Major Disease (CAMD) which facilitates interactions with corporations, regulators, research foundations, government agencies, academic experts, and perhaps most critically with patient advocacy groups.[1] Global collaborations established under CAMD involve over 1300 scientists and 61 corporations with “C-Path” consortia in over a dozen fields including Parkinson's, Alzheimer's, Duchenne's disease and multiple sclerosis. Their express goals include development of data standards, clinical outcome assessment instruments, and biomarkers. Coordination and integration of efforts will become vital if maximal benefits are to be derived from the near explosive growth in digital applications which are becoming available. Examples include the mPower application launched by Apple for PD patients together with its ResearchKit (for health-care professionals) and CareKit. Apps from academic medical centers include the iPhone-based MyHeart Counts and the Android-based Parkinson's Voice Initiative. Online communities such as PatientsLikeMe ® have been established, creating major opportunities for data sharing, but in order that such be clinically useful, some level of standardization will be necessary.


  Incentives from End-Payors? Top


Perhaps, the most critical component in a fully realized world of digital biomarker-driven health care will be the end-payors. Without reimbursable cost designation, many anticipated devices will not be economically viable and stagnation of the innovation cycle would result. That said, the prospect of millions of patients succumbing to neurodegenerative diseases in an era where life expectancy is increasing could ultimately bankrupt the managed health-care system as we know it. It is thus likely that the technology push from device manufacturers and tech-savvy patients will be met by end-user pull from the managed health-care industry and major employers. A utopic version involves providers agreeing to reduced premiums for patients who enrolled in digital health monitoring programs, and some examples are beginning to emerge (vide infra). Comparison with the automobile insurance industry is tempting, where customers of one company (Allstate Inc.,) who install a monitoring device into the OBD-II port of their automobile (Drivewise ®) may qualify for premium discounts dependent on driving habits. Some of the pioneers of this model are driving innovation in the industry. For example, WellDoc was the first company to develop an FDA approved, physician prescribed, and end-payer reimbursed digital medicine product.[60] The product, Blue Star, is a digital disease management application for subscribers with type 2 diabetes and is accessed through mobile device to assist in glucose level management. Omada Health has introduced a performance-based prevention program which is marketed directly to employer health plans. In partnership with both Kaiser Permanente and Humana, the company is reimbursed based on performance with its enrollees. Its online course (entitled prevent) lasts 4 months and targets motivated individuals who are considered at risk for diabetes and obesity. Digital sensors are provided in the form of a pedometer and weight scale, and a personal coach assists users tracking their health data. The program also provides guidance on nutrition and fitness and access to its peer community of users. AliveCor has developed mobile ECG technology which is attached to a smartphone case. Data are automatically analyzed for irregularities through an associated app which has been FDA validated through a number of clinical studies. Reportedly, from a study conducted at the Cleveland Clinic, the monitor had near total specificity for the detection of atrial fibrillation and flutter, offering an alternative to conventional monitoring. Establishing reference quality data will require longitudinal studies which incorporate physical inputs coupled with patient biorhythmic, genomic, and microbiome data. One company leading such efforts is Arivale, co-founded by Leroy Hood of the Institute for Systems Biology. Participants, identified through their 100k Challenge, provide lifestyle data, and DNA variants are correlated and tracked along with data obtained from gut microbiome, blood, and saliva samples.[61] It can be expected that increased engagement of end-payors in this process will translate to the introduction of reimbursement systems and incentives to incorporate datasets into the physicians' decision-making dashboard. According to a recent American Medical Association survey, some 26% of clinicians already actively recommend patients' use of such technology and 13% currently deploy remote monitoring protocols.[55]


  Case Studies Top


Numerous examples on the impact of digital biomarkers in the diagnosis, management, and treatment of acute and chronic diseases are being reported regularly.[62],[63],[64] The most compelling to date is in the diabetes space, where meta-analyses have demonstrated that the use of apps has resulted in statistically and clinically significant improvements in HbA1c levels in Type 2 patients.[55] The following examples are representative of clinical studies currently underway.

Cardiovascular disease

Studies are actively investigating the impact of implanted and wearable devices through clinical trials.[11] The most prominent example of application of digital biomarkers in a commercial device is the ECG developed by AliveCor Inc.[59] This class II medical device connects to a smartphone and records single-channel ECG rhythms and heart rates through patients placing fingers on the device or placing the device on the chest. Electrical signals are converted to ultrasound by the device which then transmits to the smartphone microphone.[59] Data are interpreted in real time and also sent to the cloud to allow storage, interpretation, and monitoring by a physician. The FDA-approved device is being studied in a long-term clinical trial on atrial fibrillation and had to meet exacting regulatory criteria including the standardization of cellphone technology.[65] It is also likely that studies in one therapeutic area can inform that of others. One of the most widely cited longitudinal health-care studies is the Framingham Heart Study. Now, several decades in the study are providing insightful estimates of trends in the incidence of dementia among its participants.[66] As a study of this magnitude is unlikely to be repeated, it seems likely to serve as a reference model for study power in both the cardiovascular and the neurodegenerative field and potentially a test bed for the application of digital biomarkers.

Parkinson's disease

Longitudinal evaluation of disease progression involves complex inputs and individual variations have limited the success of many trials. A goal of the Parkinson@Home study is to provide continual assessment in an at-home study, using multiple sensors linked to a smartphone application. The study, which involves a total of 1000 patients, will be used to determine patient compliance with the devices and the usefulness of the obtained data in addressing clinically relevant questions.[33] The two-phase study involves assessment of disease state using the protocol from the Parkinson's Progression Markers Initiative, and participants continually wear a designated smartwatch, fall detector, and smartphone, all linked to a custom smartphone app. Over the 90-day period of the study, sensors estimate physical activity, sleep quality, tremor, and falls. Medication intake and fall incidents will be measured through patients' self-reports in the smartphone app. The initial phase of the study is designed to validate the feasibility of the protocol, whereas in Phase 2, sensor outputs will be used to correlate clinical endpoints. The study (Phase 2 commenced late 2016) should provide proof-of-principle for the use of patient collected data and the suitability of wearable sensor technology in day-to-day living for patients in various stages of PD. In addition, the team plans to develop a database which can be used to validate data obtained from sensors in real-world settings, and the associated the algorithms behind them.[31]

Another (global) study has been established using high-frequency remote sampling from smartphones.[67] The system, Hopkins PD, involves both patient-initiated measurement of symptoms (which are prompted throughout the day) and background data obtained continuously through passive means. Five key traits are assessed voice, balance, gait, dexterity, and reaction time, and provision is made to correct pre/postadministration of medication (levodopa or other). The study commenced in 2014 using online enrollment, recruiting over 200 participants within 6 months, and is reportedly able to detect differences related to medication administration with >70% accuracy.[67]

The validation of smartphones in diagnosing and assessing PD has been investigated through a study where participants were baseline assessed, then participated in series of smartphone (android)-based analyses at home over a 1-month period.[68] Smartphone inputs were used to assess voice, posture, gait, finger tapping, and response time in a cohort of 20 participants by performing a series of tasks several times/day. Interaction with a PD specialist was provided weekly through a telemedicine appointment, who also performed a modified UPDRS. Statistical analysis was used to discriminate participants with PD from controls and predict the modified motor component of participants. Excellent correlations were observed on both counts (>95% accuracy), suggesting this to be a viable diagnostic and assessment protocol.[68]

Alzheimer's disease

In a collaborative venture between Akili Interactive Laboratories and Pfizer, positive data have been reported using Akili's digital biomarker assessment tool among participants with and without brain amyloidosis, an acknowledged Alzheimer's biomarker.[25] The tool, based on a gaming platform, is expected to allow identification of patients for subsequent positron emission tomography imaging, allowing conclusive diagnosis with commercial radiotracers.[69]

A smartpad-based cognitive assessment tool has been demonstrated in preclinical Alzheimer's patients.[24] Based around the Apple iPad system, the “C3-PAD” was deployed for at home use by older adult participants without access to a trained psychometrician. With a cohort of 49 participants, reliable and valid cognitive data were reportedly obtained from the assessments. In addition, through prior in-clinic training, the percentage of participants completing at-home assessments correctly was high, suggesting potential for future at-home cognitive testing which can be incorporated into clinical trial design.[24]

Huntington's disease

A recent pilot study was designed to determine the feasibility of at-home assessment of motor impairment in Huntington's disease patients.[39] A total of 20 participants wore multiple sensors (accelerometers) and gait measurements were taken both in a clinical setting and at home. Participants we classified according to the Unified Huntington's Disease Rating Scale total motor score using the Cohen's d value. Significant differences were observed in gait measures for the at-home group versus the in-clinic group, and additionally, it was found that the gait of those with higher total motor scores differed significantly from participants with lower total motor scores. The study suggests at-home monitoring and assessment is feasible, and variations can be attributed to clinical standards.[39]


  Conclusions and Outlook Top


There is growing confidence that the use of digital biomarkers in the diagnosis, monitoring, and ultimately treatment of diseases will have a transformative impact on managed health care. To present compelling drivers for end-payors and developers, strategically designed longitudinal studies will be necessary, which will be guided by historical and recent efforts.[66] One option would be through the national network of Clinical and Translational Science Institutes, which have capacity for multicenter trials. For example, one of its components, the PCORnet, recently participated in a summit on Alzheimer's disease and the development of digital tools.[70]

It is also likely that the application of these digital tracking tools will become integrated in areas currently in their infancy. One obvious use case would be in the development of specialized exercise equipment for patients with neuromuscular and neurodegenerative diseases. Evidence points to the beneficial impact of targeted exercise in Parkinson's patients where mobility can often be restored for a period of weeks.[71] Adding the ability to layer clinical grade digital monitoring inputs and outputs in real time could prove transformative for these and other patients and is likely to attract interest from end-payors motivated by tangible measures.

The future developments in this space will ultimately be guided by a combination of financial, regulatory, and outcomes-driven assessments. Very recent data confirm that over 800 clinical trials are currently underway on the use of digital technologies, the majority of which are in the USA.[55] In terms of technology development, more than 300,000 apps are now on the market along with 340 consumer wearable devices. This said that a number of challenges will need to be addressed in order for such technology to be adopted widely and integrated into common health-care practice.[55] The six noted barriers to introduction that have been identified by the health-care industry are (a) patient usability, (b) safety and efficacy, (c) data privacy and security, (d) malpractice risk, (e) financial incentive, and (f) clinician usability. Progress is being made in each area at a rapid pace as the financial ramifications are evident and scalability on the global stage realistic. It is also likely that development will be more rapid in select therapeutic areas. The FDA approval of the first closed-loop diabetes management system (Medtronic 670G) introduced in 2017 represents a watershed event and likely to stimulate continued advances in sensor design, data communication protocols, and drug delivery techniques. One could envision similar developments in the cardiovascular and CNS marketplace, where real-time administration of an event stabilizing therapeutic could offer immediate benefit. There remains little doubt that the digital biomarker ecosystem will evolve at a rapid rate and will remain one of the most exciting components of contemporary medicine for years to come.


  Methods Used Top


The author team conducted in-depth literature searches using Medline, PubMed, and SciFinder Scholar over the period 2012–2017. Search terms included digital biomarkers, digital medicine, m-health, and e-health. Additional searches of abstracts from conferences were conducted along with regulatory guidance documents.

Acknowledgment

The authors would like to thank William A. Jones for creating [Figure 1].

Financial support and sponsorship

GBJ acknowledges funding from the NIH National Center for Advancing Translational Sciences through grant UL1 TR001064.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Arneric SP. Assessment of Cognition & Function in Neurodegenerative Diseases in CAMD Digital Biomarker Conference: Use of Biosensors in Clinical Trials: Barriers & Solutions to the Current Landscape; 31 March, 2016. Available from: https://www.c-path.org/wp-content/uploads/2016/02/01_-_steve_arneric_camd_digital_biomarker_conference-31mar2016_-final.pdf. [Last accessed on 2017 Nov 19].  Back to cited text no. 1
    
2.
Wortley DJ. The consumerization of digital medicine. Digit Med 2016;2:135-9.  Back to cited text no. 2
  [Full text]  
3.
Dorsey ER. A digital journal for a digital era. Digit Biomark 2017;1:1-3.  Back to cited text no. 3
    
4.
Wang T, Azad T, Rajan R. The emerging influence of digital biomarkers on healthcare; 2016. Available from: https://www.rockhealth.com/reports/. [Last accessed on 2017 Nov 19].  Back to cited text no. 4
    
5.
Paone JF, Waalkes TP, Baker RR, Shaper JH. Serum UDP-galactosyl transferase as a potential biomarker for breast carcinoma. J Surg Oncol 1980;15:59-66.  Back to cited text no. 5
    
6.
Brotman B, Prince AM. Gamma-glutamyltransferase as a potential surrogate marker for detection of the non-A, non-B carrier state. Vox Sang 1988;54:144-7.  Back to cited text no. 6
    
7.
Boone CW, Kelloff GJ. Intraepithelial neoplasia, surrogate endpoint biomarkers, and cancer chemoprevention. J Cell Biochem Suppl 1993;17F: 37-48.  Back to cited text no. 7
    
8.
Amur S, LaVange L, Zineh I, Buckman-Garner S, Woodcock J. Biomarker qualification: Toward a multiple stakeholder framework for biomarker development, regulatory acceptance, and utilization. Clin Pharmacol Ther 2015;98:34-46.  Back to cited text no. 8
    
9.
Wicks P. Digital biomarkers for mood disorders. Eur Neuropsychopharmacol 2015;25:S112-3.  Back to cited text no. 9
    
10.
Boksa P. A way forward for research on biomarkers for psychiatric disorders. J Psychiatry Neurosci 2013;38:75-7.  Back to cited text no. 10
    
11.
Kramer F, Dinh W. Molecular and digital biomarker supported decision making in clinical studies in cardiovascular indications. Arch Pharm (Weinheim) 2016;349:399-409.  Back to cited text no. 11
    
12.
Bergenstal RM, Garg S, Weinzimer SA, Buckingham BA, Bode BW, Tamborlane WV, et al. Safety of a hybrid closed-loop insulin delivery system in patients with type 1 diabetes. JAMA 2016;316:1407-8.  Back to cited text no. 12
    
13.
Tucker GT. Personalized drug dosage – Closing the loop. Pharm Res 2017;34:1539-43.  Back to cited text no. 13
    
14.
Meister S, Deitersand W, Becker S. Digital health and digital biomarkers – Enabling value chains on health data. Curr Dir Biomed Eng 2016;2:577-81.  Back to cited text no. 14
    
15.
Espay AJ, Bonato P, Nahab FB, Maetzler W, Dean JM, Klucken J, et al. Technology in Parkinson's disease: Challenges and opportunities. Mov Disord 2016;31:1272-82.  Back to cited text no. 15
    
16.
Ledger D, McCaffrey D, editors. In: Inside Wearables: How the Science of Human Behavior Change offers the Secret to Long-Term Engagement. Cambridge, MA: Endeavour Partners LLC; 2014. p. 1-17.  Back to cited text no. 16
    
17.
PWC Consumer Intelligence Series: The Wearable Future. Pricewaterhouse Coopers; 2016. Available from: http://www.pwc.com/us/en/retail-consumer/publications/assets/pwc-cis-wearable-future.pdf. [Last accessed on 2017 Nov 19].  Back to cited text no. 17
    
18.
Lamkin P. Wearable. 11/5; 2015. Available from: https://www.wareable.com/fitness-trackers/adidas-micoach-fit-smart-review. [Last accessed on 2017 Nov 19].  Back to cited text no. 18
    
19.
Madan A, Moturu S, Brandis M. Method for Modeling Behavior and Health Changes. US Pat; 2014.  Back to cited text no. 19
    
20.
Jonathan E, Leahy M. Investigating a smartphone imaging unit for photoplethysmography. Physiol Meas 2010;31:N79-83.  Back to cited text no. 20
    
21.
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542:115-8.  Back to cited text no. 21
    
22.
23.
Kühnemund M, Wei Q, Darai E, Wang Y, Hernández-Neuta I, Yang Z, et al. Targeted DNA sequencing and in situ mutation analysis using mobile phone microscopy. Nat Commun 2017;8:13913.  Back to cited text no. 23
    
24.
Rentz DM, Dekhtyar M, Sherman J, Burnham S, Blacker D, Aghjayan SL, et al. The feasibility of at-home iPad cognitive testing for use in clinical trials. J Prev Alzheimers Dis 2016;3:8-12.  Back to cited text no. 24
    
25.
Akili and Pfizer Release Positive Data from Digital Biomarker Study. Available from: https://www.businesswire.com/news/home/20161209005095/en/Akili-Pfizer-Present-Positive-Data-Digital-Biomarker. [Last accessed on 2017 Nov 19].  Back to cited text no. 25
    
26.
Wortley D, An JY, Nigg CR. Wearable technologies, health and well-being: A case review. Digit Med 2017;3:11-7.  Back to cited text no. 26
  [Full text]  
27.
Tamura T, Maeda Y, Sekine M, Yoshida M. Wearable photoplethysmographic sensors – Past and present. Electronics 2014;3:282-302.  Back to cited text no. 27
    
28.
Kamshilin AA, Nippolainen E, Sidorov IS, Vasilev PV, Erofeev NP, Podolian NP, et al. Anew look at the essence of the imaging photoplethysmography. Sci Rep 2015;5:10494.  Back to cited text no. 28
    
29.
Feiner JR, Severinghaus JW, Bickler PE. Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: The effects of oximeter probe type and gender. Anesth Analg 2007;105:S18-23.  Back to cited text no. 29
    
30.
Ralston AC, Webb RK, Runciman WB. Potential errors in pulse oximetry. III: Effects of interferences, dyes, dyshaemoglobins and other pigments. Anaesthesia 1991;46:291-5.  Back to cited text no. 30
    
31.
Hsiu H, Hsu CL, Wu TL. Effects of different contacting pressure on the transfer function between finger photoplethysmographic and radial blood pressure waveforms. Proc Inst Mech Eng H 2011;225:575-83.  Back to cited text no. 31
    
32.
Bartels K, Thiele RH. Advances in photoplethysmography: Beyond arterial oxygen saturation. Can J Anaesth 2015;62:1313-28.  Back to cited text no. 32
    
33.
Silva de Lima AL, Hahn T, de Vries NM, Cohen E, Bataille L, Little MA, et al. Large-scale wearable sensor deployment in Parkinson's patients: The Parkinson@Home study protocol. JMIR Res Protoc 2016;5:e172.  Back to cited text no. 33
    
34.
Regalado A. Google's verily unveils a health watch for research. MIT Technol Rev 2017;4:14. Available from: https://www.technologyreview.com/s/604198/googles-verily -unveils-a-health-watch-for-research/#comments. [Last accessed on 2017 Nov 19].  Back to cited text no. 34
    
35.
Dorsey ER, Marks WJ Jr. Verily and its approach to digital biomarkers. Digit Biomark 2017;1:96-9.  Back to cited text no. 35
    
36.
Regalado A. I saw Alphabet's health watch. MIT Technol Rev 2016;9:28. Available from: https://www.technologyreview.com/s/602466/i-saw-alphabets-health-watch/. [Last accessed on 2017 Nov 19].  Back to cited text no. 36
    
37.
Das C, Chatterjee S, Kumaraswamy G, Krishnamoorthy K. Elastic compressible energy storage devices from ICE templated polymer gels treated with polyphenols. J Phys Chem C 2017;121:3270-8.  Back to cited text no. 37
    
38.
Rissanen A, Saari H. 2014. Available from: http://www.vttresearch.com/Impulse/Pages/Fabry-Perot-Interferometer-technologies.aspx. [Last accessed on 2017 Nov 19].  Back to cited text no. 38
    
39.
Andrzejewski KL, Dowling AV, Stamler D, Felong TJ, Harris DA, Wong C, et al. Wearable sensors in Huntington disease: A Pilot study. J Huntingtons Dis 2016;5:199_206.  Back to cited text no. 39
    
40.
Farandos NM, Yetisen AK, Monteiro MJ, Lowe CR, Yun SH. Contact lens sensors in ocular diagnostics. Adv Healthc Mater 2015;4:792-810.  Back to cited text no. 40
    
41.
von Thun Und Hohenstein-Blaul N, Funke S, Grus FH. Tears as a source of biomarkers for ocular and systemic diseases. Exp Eye Res 2013;117:126-37.  Back to cited text no. 41
    
42.
Kalló G, Emri M, Varga Z, Ujhelyi B, Tőzsér J, Csutak A, et al. Changes in the chemical barrier composition of tears in Alzheimer's disease reveal potential tear diagnostic biomarkers. PLoS One 2016;11:e0158000.  Back to cited text no. 42
    
43.
Lim YY, Johnson LN, Fernandez BM, Santos CY, Maruff P, Snyder PJ. High-resolution retinal imaging in the identification of preclinical Alzheimer's disease. Alzheimers Demen 2015;11:S158-9.  Back to cited text no. 43
    
44.
Doyle FJ 3rd, Huyett LM, Lee JB, Zisser HC, Dassau E. Closed-loop artificial pancreas systems: Engineering the algorithms. Diabetes Care 2014;37:1191-7.  Back to cited text no. 44
    
45.
Klopfleisch R, Jung F. The pathology of the foreign body reaction against biomaterials. J Biomed Mater Res A 2017;105:927-40.  Back to cited text no. 45
    
46.
Wang Y, Vaddiraju S, Gu B, Papadimitrakopoulos F, Burgess DJ. Foreign body reaction to implantable biosensors: Effects of tissue trauma and implant size. J Diabetes Sci Technol 2015;9:966-77.  Back to cited text no. 46
    
47.
Wang Y, Papadimitrakopoulos F, Burgess DJ. Polymeric “smart” coatings to prevent foreign body response to implantable biosensors. J Control Release 2013;169:341-7.  Back to cited text no. 47
    
48.
Zhukov I, Mikhaylov D, Starikovskiy A. Nano sensors integrated into dental implants for detection of acute myocardial infarction. Int J Emerg Trends Technol Comput Sci 2012;1:85-7.  Back to cited text no. 48
    
49.
Cash KJ, Clark HA. Nanosensors and nanomaterials for monitoring glucose in diabetes. Trends Mol Med 2010;16:584-93.  Back to cited text no. 49
    
50.
Bennett MG, Naranja RJ Jr. Getting nano tattoos right – A checklist of legal and ethical hurdles for an emerging nanomedical technology. Nanomedicine 2013;9:729-31.  Back to cited text no. 50
    
51.
Gibney E. The body electric. Nature 2015;528:26-8.  Back to cited text no. 51
    
52.
Koh A, Kang D, Xue Y, Lee S, Pielak RM, Kim J, et al. Asoft, wearable microfluidic device for the capture, storage, and colorimetric sensing of sweat. Sci Transl Med 2016;8:366ra165.  Back to cited text no. 52
    
53.
Kane JM, Perlis RH, DiCarlo LA, Au-Yeung K, Duong J, Petrides G, et al. First experience with a wireless system incorporating physiologic assessments and direct confirmation of digital tablet ingestions in ambulatory patients with schizophrenia or bipolar disorder. J Clin Psychiatry 2013;74:e533-40.  Back to cited text no. 53
    
54.
Rohatagi S, Profit D, Hatch A, Zhao C, Docherty JP, Peters-Strickland TS, et al. Optimization of a digital medicine system in psychiatry. J Clin Psychiatry 2016;77:e1101-e1107.  Back to cited text no. 54
    
55.
Aitken M, Clancy B, Nass D. The Growing Value of Digital Health Evidence and Impact on Human Health and the Healthcare System. Iqvia Institute; 7 November 2017. Available from: https://www.iqvia.com/institute/reports/the-growing-value-of-digital-health. [Last accessed on 2017 Nov 19].  Back to cited text no. 55
    
56.
FDA 2016: Principles for Codevelopment of an in vitro Companion Diagnostic Device with a Therapeutic Product: Draft Guidance for Industry and Food and Drug Administration Staff; 15 July, 2016. Available from: https://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM510824.pdf. In Vitro Companion Diagnostic Devices: Guidance for Industry and Food and Drug Administration Staff; 6 August, 2014. Available from: https://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM262327.pdf. and: FDA-NIH Biomarker Working Group. BEST Biomarkers, Endpoints, and other Tools Resource. Silver Spring (MD), Bethesda (MD): Food and Drug Administration (US), National Institutes of Health (US); 2016. Available from: https://www.ncbi.nlm.nih.gov/books/NBK326791/. [Last accessed on 2017 Nov 19].  Back to cited text no. 56
    
57.
FDA 2015: General Wellness: Policy for Low Risk Devices Guidance for Industry and Food and Drug Administration Staff. Issued on 1/20/15 and in Final Form on 7/29/16. Available from: https://www.fda.gov/medicaldevices/digitalhealth/. [Last accessed on 2017 Nov 19].  Back to cited text no. 57
    
58.
59.
Vashist SK, Schneider EM, Luong JH. Commercial smartphone-based devices and smart applications for personalized healthcare monitoring and management. Diagnostics (Basel) 2014;4:104-28.  Back to cited text no. 59
    
60.
Steinberg D, Horwitz G, Zohar D. Building a business model in digital medicine. Nat Biotechnol 2015;33:910-20.  Back to cited text no. 60
    
61.
Hood L, Price ND. Promoting wellness and demystifying disease: The 100K wellness project. Clin Omics 2014;3:20-4.  Back to cited text no. 61
    
62.
Adams JL, Dinesh K, Xiong M, Tarolli CG, Sharma S, Sheth N, et al. Multiple wearable sensors in Parkinson and Huntington disease individuals: A pilot study in clinic and at home. Digit Biomark 2017;1:52-63.  Back to cited text no. 62
    
63.
Heldman DA, Giuffrida JP, Cubo E. Wearable sensors for advanced therapy referral in Parkinson's disease. J Parkinsons Dis 2016;6:631-8.  Back to cited text no. 63
    
64.
Giggins OM, Clay I, Walsh L. Physical activity monitoring in patients with neurological disorders: A review of novel body-worn devices. Digit Biomark 2017;1:14-42.  Back to cited text no. 64
    
65.
Hickey KT, Hauser NR, Valente LE, Riga TC, Frulla AP, Masterson Creber R, et al. Asingle-center randomized, controlled trial investigating the efficacy of a mHealth ECG technology intervention to improve the detection of atrial fibrillation: The iHEART study protocol. BMC Cardiovasc Disord 2016;16:152.  Back to cited text no. 65
    
66.
Satizabal CL, Beiser AS, Chouraki V, Chêne G, Dufouil C, Seshadri S, et al. Incidence of dementia over three decades in the Framingham heart study. N Engl J Med 2016;374:523-32.  Back to cited text no. 66
    
67.
Zhan A, Little MA, Harris DA, Abiola SO, Dorsey ER, Saria S, et al. High frequency remote monitoring of Parkinson's disease via smartphone: Platform overview and medication response detection. Eprint ArXiv 2016.  Back to cited text no. 67
    
68.
Abdolahi A, Scoglio N, Killoran A, Dorsey ER, Biglan KM. Potential reliability and validity of a modified version of the unified Parkinson's disease rating scale that could be administered remotely. Parkinsonism Relat Disord 2013;19:218-21.  Back to cited text no. 68
    
69.
Landau SM, Thomas BA, Thurfjell L, Schmidt M, Margolin R, Mintun M, et al. Amyloid PET imaging in Alzheimer's disease: A comparison of three radiotracers. Eur J Nucl Med Mol Imaging 2014;41:1398-407.  Back to cited text no. 69
    
70.
71.
Wrann CD. FNDC5/Irisin – their role in the nervous system and as a mediator for beneficial effects of exercise on the brain. Brain Plast 2015;1:55-61.  Back to cited text no. 71
    


    Figures

  [Figure 1]


This article has been cited by
1 Impact of individual and treatment characteristics on wearable sensor-based digital biomarkers of opioid use
Brittany P. Chapman, Bhanu Teja Gullapalli, Tauhidur Rahman, David Smelson, Edward W. Boyer, Stephanie Carreiro
npj Digital Medicine. 2022; 5(1)
[Pubmed] | [DOI]
2 Beyond Digital Twins: Phygital Twins for Neuroergonomics in Human-Robot Interaction
Giacinto Barresi, Claudio Pacchierotti, Matteo Laffranchi, Lorenzo De Michieli
Frontiers in Neurorobotics. 2022; 16
[Pubmed] | [DOI]
3 Role of Wearable Sensing Technology to Manage Long COVID
Kamil Reza Khondakar, Ajeet Kaushik
Biosensors. 2022; 13(1): 62
[Pubmed] | [DOI]
4 Meaningful change: Defining the interpretability of changes in endpoints derived from interactive and mHealth technologies in healthcare and clinical research
B Byrom, P Breedon, R Tulkki-Wilke, JV Platko
Journal of Rehabilitation and Assistive Technologies Engineering. 2020; 7: 2055668319
[Pubmed] | [DOI]
5 Dem MS-Phänotyp auf der Spur
Tjalf Ziemssen
DNP - Der Neurologe & Psychiater. 2019; 20(5): 33
[Pubmed] | [DOI]
6 Emergence of digital biomarkers to predict and modify treatment efficacy: machine learning study
Nicole L Guthrie,Jason Carpenter,Katherine L Edwards,Kevin J Appelbaum,Sourav Dey,David M Eisenberg,David L Katz,Mark A Berman
BMJ Open. 2019; 9(7): e030710
[Pubmed] | [DOI]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Introduction
The Clinical Val...
Devices
Regulatory Pathways
The Patient-Cent...
Incentives from ...
Case Studies
Conclusions and ...
Methods Used
References
Article Figures

 Article Access Statistics
    Viewed8032    
    Printed330    
    Emailed0    
    PDF Downloaded765    
    Comments [Add]    
    Cited by others 6    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]