2017-2018 Assessment of The Army Research Laboratory PDF

THE NATIONAL ACADEMIES PRESS
This PDF is available at http://nap.edu/25011 SHARE

   
2017-2018 Assessment of the Army Research Laboratory:

Interim Report
DETAILS
168 pages | 8.5 x 11 | PAPERBACK

ISBN 978-0-309-47161-9 | DOI 10.17226/25011
CONTRIBUTORS
GET THIS BOOK Army Research Laboratory Technical Assessment Board; Laboratory Assessments
Board; Division on Engineering and Physical Sciences; National Academies of
Sciences, Engineering, and Medicine
FIND RELATED TITLES

Visit the National Academies Press at NAP.edu and login or register to get:
– Access to free PDF downloads of thousands of scientiﬁc reports

– 10% off the price of print titles
– Email or social media notiﬁcations of new titles related to your interests
– Special offers and discounts

Distribution, posting, or copying of this PDF is strictly prohibited without written permission of the National Academies Press.
(Request Permission) Unless otherwise indicated, all materials in this PDF are copyrighted by the National Academy of Sciences.
Copyright © National Academy of Sciences. All rights reserved.

2017-2018 Assessment of the Army Research Laboratory: Interim Report
Army Research Laboratory Technical Assessment Board
Laboratory Assessments Board
Division on Engineering and Physical Sciences
A Consensus Study Report of
Copyright National Academy of Sciences. All rights reserved.

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by Contract No. W911QX-14-C-0039 from the Army Research Laboratory.
Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily
reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-47161-9

International Standard Book Number-10: 0-309-47161-3
Digital Object Identifier: https://doi.org/10.17226/25011
Additional copies of this report are available for sale from the National Academies Press, 500 Fifth Street,
NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2018 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2018. 2017-2018
Assessment of the Army Research Laboratory: Interim Report. Washington, DC: The National Academies
Press. https://doi.org/10.17226/25011.

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by
President Lincoln, as a private, nongovernmental institution to advise the nation on issues
related to science and technology. Members are elected by their peers for outstanding
contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the
National Academy of Sciences to bring the practices of engineering to advising the nation.
Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D.
Mote, Jr., is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in
1970 under the charter of the National Academy of Sciences to advise the nation on medical
and health issues. Members are elected by their peers for distinguished contributions to
medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering,
and Medicine to provide independent, objective analysis and advice to the nation and
conduct other activities to solve complex problems and inform public policy decisions. The
National Academies also encourage education and research, recognize outstanding
contributions to knowledge, and increase public understanding in matters of science,
engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at
www.nationalacademies.org.

Consensus Study Reports published by the National Academies of Sciences, Engineering, and
Medicine document the evidence-based consensus on the study’s statement of task by an
authoring committee of experts. Reports typically include findings, conclusions, and
recommendations based on information gathered by the committee and the committee’s
deliberations. Each report has been subjected to a rigorous and independent peer-review
process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and

Medicine chronicle the presentations and discussions at a workshop, symposium, or
other event convened by the National Academies. The statements and opinions contained in
proceedings are those of the participants and are not endorsed by other participants, the
planning committee, or the National Academies.
For information about other products and activities of the National Academies, please
visit www.nationalacademies.org/about/whatwedo.

ARMY RESEARCH LABORATORY TECHNICAL ASSESSMENT BOARD
JENNIE S. HWANG, NAE,1 H-Technologies Group, Chair

MARK EBERHART, Colorado School of Mines
GEORGE (RUSTY) T. GRAY III, NAE, Los Alamos National Laboratory
PRABHAT HAJELA, Rensselaer Polytechnic Institute
WESLEY L. HARRIS, NAE, Massachusetts Institute of Technology
WILLIAM S. MARRAS, NAE, Ohio State University
ALAN NEEDLEMAN, NAE, Texas A&M University
DANIEL A. REED, University of Iowa
Staff
AZEB GETACHEW, Senior Program Assistant

EVA LABRE, Administrative Coordinator
JAMES P. MCGEE, Director
ARUL MOZHI, Senior Program Officer
PANEL ON ASSESSMENT AND ANALYSIS

AT THE ARMY RESEARCH LABORATORY
ALAN NEEDLEMAN, NAE, Texas A&M University, Chair

JAMES P. BAGIAN, NAE/NAM,2 University of Michigan
CAMERON R. (DALE) BASS, Duke University
JOHN M. CAVANAUGH, Wayne State University
ELLIOT L. CHAIKOF, NAM, Harvard Medical School
STEPHEN C. MERRIMAN, SCMerriman Consulting, LLC; Boeing Associate Technical Fellow (retired)
JAMES F. O’BRYON, Department of Defense (retired), The O’Bryon Group
FREDERICK P. RIVARA, NAM, University of Washington School of Medicine
JILL H. SMITH, U.S. Army Communications-Electronics RDE Center (retired); Executive Consultant
JAMES J. STREILEIN, MITRE
PANEL ON BALLISTICS SCIENCE AND ENGINEERING

GEORGE (RUSTY) T. GRAY III, NAE, Los Alamos National Laboratory, Chair
CHARLES E. ANDERSON, JR., CEA Consulting
MELVIN R. BAER, Sandia National Laboratories (retired)
ROMESH C. BATRA, Virginia Polytechnic Institute and State University
JAMES A. COOPER, JR., Musculoskeletal and Translational Tissue Engineering Research (MATTER)
MARCIA A. COOPER, Sandia National Laboratories
ROGER GHANEM, University of Southern California
ZELDA GILLS, Lockheed Martin Aeronautics Company
MICHAEL JAFFE, New Jersey Institute of Technology
JERRY A. KRILL, Johns Hopkins University Applied Physics Laboratory
1
Member, National Academy of Engineering.
2
Member, National Academy of Medicine.

ALLAN T. MENSE, Raytheon Missile Systems

DONALD PROSNITZ, Consultant
FRANK J. SERNA, Charles Stark Draper Laboratory
LESLIE E. SMITH, National Institute of Standards and Technology (retired)
STEVEN F. SON, Purdue University
GHATU SUBHASH, University of Florida
PANEL ON COMPUTATIONAL SCIENCES

DANIEL A. REED, University of Iowa, Chair

TILAK AGERWALA, IBM (retired), TKMA Consulting, LLC
RALPH C. ALDREDGE III, University of California, Davis
WILLIAM D. GROPP, NAE, University of Illinois, Urbana-Champaign
JAMES T. KAJIYA, NAE, Microsoft Corporation (retired); Tolt Machine Works, Inc.
GEORGE KARYPIS, University of Minnesota
PETER M. KOGGE, University of Notre Dame
VIPIN KUMAR, University of Minnesota
JAMES L. MCLELLAND, NAS,1 Stanford University
KYRAN D. MISH, Sandia National Laboratories
LINDA A. NESS, Independent Consultant
KALYAN PERUMALLA, Oak Ridge National Laboratory
PADMA RAGHAVAN, Vanderbilt University
GUY LEWIS STEELE, JR., NAE, Oracle Labs
PANEL ON HUMAN FACTORS SCIENCE

WILLIAM S. MARRAS, NAE, Ohio State University, Chair

FREDERICK R. CHANG, NAE, Southern Methodist University
LORRIE FAITH CRANOR, Carnegie Mellon University
CHARLES A. CZEISLER, NAM, Harvard Medical School
BARBARA A. DOSHER, NAS, University of California, Irvine
JEANNE F. DUFFY, Harvard Medical School; Brigham and Women’s Hospital
DAVID J. HEEGER, NAS, New York University
WILLIAM W. LYTTON, SUNY Downstate Medical Center
KATHERINE L. MORSE, Johns Hopkins University Applied Physics Laboratory
SHRIKANTH (SHRI) S. NARAYANAN, University of Southern California
STEPHEN F. SANDS, Sands Research, Inc.
ANITA WILLIAMS WOOLLEY, Carnegie Mellon University
1
Member, National Academy of Sciences.
vi

PANEL ON INFORMATION SCIENCE

PRABHAT HAJELA, Rensselaer Polytechnic Institute, Chair

JEAN M. ANDINO, Arizona State University
ELLEN J. BASS, Drexel University
KATHLEEN M. CARLEY, Carnegie Mellon University
EDWARD F. CRAWLEY, NAE, Massachusetts Institute of Technology
MARK E. DAVIS, Medavis Consulting
SUSAN T. DUMAIS, NAE, Microsoft Research
JONATHAN GRUDIN, Microsoft Research
VERLIN B. HINSZ, North Dakota State University
JOHN M. LANICCI, Independent Atmospheric and Climate Science Consultant
J. GREGORY MORRISETT, Cornell University
RANDOLPH L. MOSES, Ohio State University
PETER WILLETT, University of Connecticut
PANEL ON MATERIALS SCIENCE AND ENGINEERING

MARK E. EBERHART, Colorado School of Mines, Chair

RICHARD C. ALKIRE, NAE, University of Illinois, Urbana-Champaign
MICHAEL I. BASKES, NAE, Mississippi State University; University of California, San Diego; Los
Alamos National Laboratory
PAUL V. BRAUN, University of Illinois, Urbana-Champaign
STEPHEN CREAGER, Clemson University
MICHAEL ETTENBERG, NAE, Dolce Technologies
OLIVIA A. GRAEVE, University of California, San Diego
JAMES S. HARRIS, NAE, Stanford University
DOUGLAS C. HOPKINS, North Carolina State University
THOMAS L. KOCH, NAE, University of Arizona
SUBHASH MAHAJAN, NAE, University of California, Davis
CHRISTIAN MAILHIOT, Sandia National Laboratories
JOSEPH W. ORENSTEIN, University of California, Berkeley
C. KUMAR N. PATEL, NAS/NAE, Pranalytica, Inc.
RAJEEV J. RAM, Massachusetts Institute of Technology
PANEL ON MECHANICAL SCIENCE AND ENGINEERING

WESLEY L. HARRIS, NAE, Massachusetts Institute of Technology, Chair

MONICA ANDERSON, University of Alabama
PATRICK J. FLYNN, University of Notre Dame
MOSHE KAM, New Jersey Institute of Technology
LYNNE E. PARKER, University of Tennessee, Knoxville
ARMANDO A. RODRIGUEZ, Arizona State University
CHARLES E. THORPE, Clarkson University
PATRICIO A. VELA, Georgia Institute of Technology
ALAN R. WAGNER, Pennsylvania State University
HOLLY YANCO, University of Massachusetts, Lowell
vii

LABORATORY ASSESSMENTS BOARD
ROSS B. COROTIS, NAE, University of Colorado, Boulder, Chair

C. WILLIAM GEAR, NAE, Princeton University, University of Illinois, Urbana-Champaign
WESLEY L. HARRIS, NAE, Massachusetts Institute of Technology
JENNIE S. HWANG, NAE, Case Western Reserve University; H-Technologies Group
W. CARL LINEBERGER, NAS, JILA,1 University of Colorado, Boulder
C. KUMAR N. PATEL, NAS/NAE, University of California, Los Angeles; Pranalytica, Inc.
ELSA REICHMANIS, NAE, Georgia Institute of Technology
LYLE H. SCHWARTZ, NAE, University of South Florida, Tampa
Staff
AZEB GETACHEW, Senior Program Assistant

LIZA HAMILTON, Associate Program Officer
EVA LABRE, Administrative Coordinator
JAMES P. MCGEE, Director
ARUL MOZHI, Senior Program Officer
1
JILA, formerly known as the Joint Institute for Laboratory Astrophysics, is a physical science research
institute located on the University of Colorado Boulder campus. JILA is jointly operated by the University of
Colorado and the National Institute of Standards and Technology.
viii

Acknowledgment of Reviewers
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse
perspectives and technical expertise. The purpose of this independent review is to provide candid and
critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in
making each published report as sound as possible and to ensure that it meets the institutional standards
for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft
manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report:
Amy E. Duwel, The Charles Stark Draper Laboratory, Inc.,

Linda Ng Boyle, University of Washington,
Mary Jane Irwin, NAE,1 Pennsylvania State University,
Arogyaswami J. Paulraj, NAE, Stanford University,
Guruswami Ravichandran, NAE, California Institute of Technology,
Michael Salkind, Indus International, Inc., and
Elihu Zimet, Gaithersburg, Maryland.
Although the reviewers listed above provided many constructive comments and suggestions, they
were not asked to endorse the conclusions or recommendations of this report nor did they see the final
draft before its release. The review of this report was overseen by David E. Crow, NAE, University of
Connecticut. He was responsible for making certain that an independent examination of this report was
carried out in accordance with the standards of the National Academies and that all review comments
were carefully considered. Responsibility for the final content rests entirely with the authoring committee
and the National Academies.
1
Member, National Academy of Engineering.
ix


Contents
SUMMARY 1
1 INTRODUCTION 12
The Biennial Assessment Process, 12
Preparation and Organization of This Report, 13
Assessment Criteria, 14
Approach Taken During Report Preparation, 15
Report Content, 16
2 MATERIALS RESEARCH 17
Energy-Efficient Electronics and Photonics, 17
Materials for Soldier and Platform Power Systems, 23
Quantum Sciences, 33
Overall Quality of the Work, 36
Conclusions and Recommendation, 36
3 SCIENCES FOR LETHALITY AND PROTECTION 38

Battlefield Injury Mechanisms, 38
Directed Energy, 45
Penetration, Armor, and Adaptive Protection, 47
Conclusions and Recommendations, 52
4 INFORMATION SCIENCES 54
Sensing and Effecting, 55
System Intelligence and Intelligent Systems, 58
Human and Information Interaction, 62
Atmospheric Sciences, 69
5 COMPUTATIONAL SCIENCES 78
Advanced Computing Architectures, 78
Data-Intensive Sciences, 81
Predictive Sciences, 84
6 SCIENCES FOR MANEUVER 89

Intelligence and Control, 89
Machine-Human Interaction, 95
Perception, 98
xi

7 HUMAN SCIENCES 105

Real-World Behavior, 105
Human Variability, 109
Humans in Multiagent Systems, 112
Human Cyber Performance, 114
8 ANALYSIS AND ASSESSMENT 123

Ballistics Survivability, Vulnerability, and Lethality, 123
Personnel Survivability, 126
Human Systems Integration, 129
9 CROSSCUTTING CONCLUSIONS AND RECOMMENDATIONS AND EXCEPTIONAL

ACCOMPLISHMENTS 134
Crosscutting Conclusions and Recommendations, 134
Exceptional Accomplishments, 137
APPENDIXES
A Army Research Laboratory Organization and Science and Technology Campaign Framework 143
B Army Research Laboratory Technical Assessment Board Members and Staff Biographical
Information 146
C Assessment Criteria 151
D Acronyms 153
xii

Summary
The statement of task that guided the work of the Army Research Laboratory Technical Assessment
Board (ARLTAB) is as follows:
An ad hoc committee to be named the Army Research Laboratory Technical Assessment Board
(ARLTAB), to be overseen by the Laboratory Assessments Board, will be appointed to continue the
function of providing biennial assessments of the scientific and technical quality of the Army Research
Laboratory (ARL). These assessments will include findings and recommendations related to the
quality of ARL’s research, development, and analysis programs. While the primary role of the
ARLTAB is to provide peer assessment, it may offer advice on related matters when requested by the
ARL Director. The ARLTAB will provide an interim assessment report at the end of Year 1 of each 2-
year assessment cycle and a final assessment report biennially. The ARLTAB will be assisted by up to
seven separately appointed panels that will focus on particular portions of the ARL program. Each
year, up to three additional panels may be appointed to assess special topics, at the request of the ARL
Director.
During the 2017-2018 assessment, the ARLTAB is being assisted by seven panels, each of which
focuses on a portion of the ARL program conducted in ARL’s science and technology (S&T) campaigns:
Materials Research, Sciences for Lethality and Protection, Information Sciences, Computational Sciences,
Sciences for Maneuver, Human Sciences, and Analysis and Assessment.
This interim report summarizes the findings of the Board for the first year of this biennial assessment;
the current report addresses approximately half the portfolio for each campaign; the remainder will be
assessed in 2018. It should be noted that because a full spectrum of projects and programs within each
ARL campaign and the interrelated mapping across all campaigns’ projects and programs was not
provided to the ARLTAB, this report represents the Board’s assessment on only the projects and
programs presented.
During the first year, the Board examined the following elements within the ARL S&T campaigns:
(1) Materials Research—energy-efficient electronics and photonics, materials for soldier and platform
power systems, and quantum sciences; (2) Sciences for Lethality and Protection—battlefield injury
mechanisms, directed energy, and penetration, armor, and adaptive protection; (3) Information
Sciences—sensing and effecting, system intelligence and intelligent systems, human and information
interaction, and atmospheric sciences; (4) Computational Sciences—advanced computing architectures,
data-intensive sciences, and predictive sciences; (5) Sciences for Maneuver—intelligence and control,
machine-human interaction, and perception; (6) Human Sciences—humans in cybersecurity, humans in
multiagent systems, human variability, and real-world behavior; and (7) Analysis and Assessment—
ballistics survivability lethality and vulnerability, personnel survivability, and human systems integration.
A second, final report will subsume the findings of this interim report and add the findings from the
second year of the review, during which the Board will examine additional elements within the ARL S&T
campaigns.
The mission of ARL, as the U.S. Army’s corporate laboratory, is to discover, innovate, and transition
science and technology to ensure dominant strategic land power. In 2013 ARL restructured its portfolio of
ongoing and planned research and development to align with its S&T campaign plans for 2015-2035.
ARL has maintained its organizational structure, consisting of six directorates: Computational and
Information Sciences Directorate (CISD), Human Research and Engineering Directorate (HRED),

Sensors and Electron Devices Directorate (SEDD), Survivability and Lethality Analysis Directorate
(SLAD), Vehicle Technology Directorate (VTD), and Weapons and Materials Research Directorate
(WMRD). The research portfolio has been organized into science and technology campaigns, each of
which describes related work supported by staff from multiple directorates. Appendix Table A.1 shows
the directorates that supported each campaign during the 2017 review. The ARL technical strategy
document describes the portfolio of each campaign in detail.1 ARL’s vision is compelling and raises
expectations for an innovative program of research designed to be responsive to the needs of the “Army
after next.” This is not yet fully evident in the portfolio currently being assessed. The reorganization of
the portfolio into key focused campaigns is promising, but it may take some time to transform and mature
the program of work to consistently align with new critical paths.
In general, the quality of the research presented, the capabilities of the leadership, the knowledge and
abilities of the investigators, and proposed future directions continue to improve. Significant gains were
evident in publication rates, numbers of postdoctoral researchers, and collaborations with relevant peers
outside ARL. The research work environments were impressive in terms of their unique and advanced
technology capabilities to support research. Overall, these are all outstanding accomplishments and mark
an advance over prior years.
MATERIALS RESEARCH
ARL’s materials sciences span the spectrum of technology maturity and address Army applications,
working from the state of the art to the art of the possible—“25 years out”—according to the ARL.
Materials research efforts and expertise are spread throughout the ARL enterprise. As the ensemble of the
materials discipline and capabilities, the area of materials sciences is one of ARL’s primary core technical
competencies. In the larger context, the mission of ARL, as the U.S. Army’s corporate laboratory, is to
discover, innovate, and transition science and technology to ensure dominant strategic land power.
Energy-Efficient Electronics and Photonics
Energy-efficient electronics and photonics are intended to address the size, weight, power, cost, and
time (SWAPCT) of soldier technologies on the battlefield. The impact of advances utilizing optical
equivalents, efficiencies realized through new radio frequency (RF) waveform and encoding strategies,
and efficiencies for directed-energy applications are envisioned as being significant and important targets.
Examples include escalation of electronic warfare technologies down to the individual unit and soldier in
a continually contested RF environment, exacerbating current power challenges even more.
Materials for Soldier and Platform Power Systems
The research programs of materials for soldier and platform power systems (MS&PP) are motivated
by soldier battlefield power needs both currently and in the future. The research supports the tactical unit
energy independence (TUEI) essential research area (ERA), with focus on unburdening the soldier by
making power lightweight, providing power on-site, and diminishing power needs, all essential enabling
factors in supporting soldier welfare and effectiveness. Toward these ends, ARL is among the country’s
top-tier research organizations. Its research portfolio includes a mixture of world-leading, established,
innovative projects and recently initiated programs anticipating scientific trends.
1
U.S. Army Research Laboratory, Army Research Laboratory Technical Strategy 2015-2035, Adelphi, Md.,
2014, https://www.arl.army.mil/www/pages/172/docs/ARL_Technical_Strategy_FINAL.pdf.

Quantum Sciences
Quantum sciences is a new program area of high scientific quality and well aligned with the long-
term goals of ARL’s mission to provide the Army of the future with clear tactical advantage. It is
anticipated that quantum sciences will provide game-changing capabilities for command, control,
communications, computers, intelligence, surveillance and reconnaissance (C4ISR) for the Army of the
future. It is critical that ARL maintain and expand this research effort.
SCIENCES FOR LETHALITY AND PROTECTION
ARL’s research in the area of sciences for lethality and protection during 2017 ranged from basic
research that improves our fundamental understanding of the scientific phenomena and technology
generation that supports battlefield injury mechanisms in human response to threats and human protective
equipment, directed energy programs, and programs that address weapon-target interactions and armor
and adaptive protection developments to benefit the warfighter.
Battlefield Injury Mechanisms
The study of battlefield injury mechanisms are a relatively new area of research at ARL, and ARL has
shown greatly improved coordination and focus over the past two years. Excellent progress on many
topics was observed during the review. There has been considerable improvement in prioritization of
projects, at least in the near term; while some of the specific goals and timelines of the remaining
elements of the program need to be better developed and articulated. The ARL focus on the definition of
biological injury in materials and engineering relevant terms is a necessary step in moving this critical
area forward and may be unique in the field. The focus at ARL on identifying the critical size scale of
injury is correct and the group emphasis on the translation of animal data to humans is necessary and
positive.
Directed Energy
The ARL directed energy (DE) program focuses on radio frequency (RF)-DE and laser-DE. ARL
leadership and research teams successfully implemented some of the recommendations of the previous
ARLTAB report.2 Specifically, the collection of presented projects demonstrated a coordinated strategy
across the enterprise for the laser-related work, with indications of much greater collaboration with the
Navy and Air Force. Internal to ARL, principal investigators (PIs) demonstrated greater awareness of
work done with threat warning and countermeasures. The quality of the programs will continue to benefit
from even deeper and more frequent collaborations both internal and external to ARL to foster rapid
innovation with operational and contextual relevance.
2
National Academies of Sciences, Engineering, and Medicine, 2017, 2015-2016 Assessment of the Army
Research Laboratory, The National Academies Press, Washington, D.C.

Penetration, Armor, and Adaptive Protection
ARL continues to demonstrate a strong record of achievement in the fundamental and applied
sciences and the engineering of penetration, armor, and adaptive protection. The ongoing work continues
to highlight how ARL is building on its history of excellence to provide the knowledge basis for future
Army needs in the area of warfighter protection. This is an absolutely critical and core competency that
underlies Army capabilities.
A start has been made in the uncertainty quantification (UQ) area of research, but there is a long way
to go. It was unclear what the level of effort of research is or the number of people involved. ARL needs
to continue an emphasis in UQ. Further, ARL needs to pursue the integration of UQ into its data-to-
decision workflow that includes modeling and simulation, experimentation, and design. To accelerate
integration and given the complexity of their objectives, ARL scientists and engineers need to leverage
software and methodologies developed at other Department of Energy (DOE) and Department of Defense
(DOD) laboratories.
INFORMATION SCIENCES
ARL research in information sciences is focused on developing and enhancing science and
technology (S&T) capabilities that allow for the timely acquisition and use of high-quality information
and knowledge at the tactical edge, for both strategic operations planning and mission deployment.
Included in this approach are technological advances that support information acquisition, reasoning with
such information, and support for decision-making activities such as collaborative communications.
Sensing and Effecting
Research projects covered thematic areas of nonimaging sensors (acoustic, electric, magnetic,
seismic), radar sensing and signal processing, image and video analytics, sensor and data fusion, and
machine learning. Noteworthy programs include electric and magnetic field sensing, research on the next-
generation improvised explosive device (IED) and landmine detection platform, computational advances
in electric field modeling, cross-modal face recognition, along with the development and dissemination of
a cross-modal face recognition data set to the academic research community, and innovative approaches
to fuse textual context with image features to improve machine learning of human activity.
The work in sensing and effecting (S&E) was assessed to be generally of high scientific quality, and
there was a balance between theoretical and experimental work, as well as evidence of transition into
practice or use by other areas.
System Intelligence and Intelligent Systems
System intelligence and intelligent systems (SIIS) research spans areas of information understanding,
information fusion, and computational intelligence. This research has produced key results in areas
relevant to Army needs, including the understanding and analysis of complex environments and streaming
data, navigation, exploration, and mapping of the physical world. The work on unsupervised learning of
semantic labels in streaming data, and the synergies between visual analysis and efficient exploration of
environments is noteworthy. It has identified gaps in the state of the art and is being disseminated in
leading venues. The approach of collaborating with researchers throughout ARL as well as on the outside
to develop a continuum of work, ranging from information analysis (in SIIS) to decision support (in
human and information interaction), will yield good dividends.

Human and Information Interaction
Human and information interaction (HII) is a new program in the Information Sciences Campaign,
and has been in operation for about one year, bringing together researchers from disparate disciplines and
technical backgrounds. “Interaction” is what distinguishes HII in the information science and human
science research space. The objective of HII research at ARL is to develop models, methods, and
understanding of data and information generated by humans and intelligent agents in a complex, multi-
genre network environment. It further examines tools to respond to user information needs with due
consideration of user variability and mission constraints, and thereby to develop timely and accurate
situational understanding.
Atmospheric Sciences
The research portfolio of the Battlefield Environments Division (BED) seeks to improve
environmental understanding of the planetary boundary layer (PBL) and processes that operate on small
spatial and temporal scales, and on developing appropriate environmental intelligence tools for deployed
soldiers to use in austere, complex operating environments. The research projects reviewed by the panel
during this cycle included detection and characterization of chemical aerosols, acoustic and infrasound
sensing, development and fielding of a meteorological sensor array at White Sands Missile Range
(WSMR), and advances in small-scale atmospheric model development, verification, and validation.
COMPUTATIONAL SCIENCES
The computational sciences panel examined projects in advanced computing architectures, data
intensive sciences (AI and machine learning), and predictive sciences. In each area, there has been
substantial progress since the last review.
Advanced Computing Architectures
Alongside operating and managing high-performance computing (HPC) systems to serve the
processing needs of the broader DoD community, the group has evolved to focus on tactical HPC at the
edge, in order to place innovative new systems at the points of need to support the soldier in complex
operational environments. The research portfolio comprises projects for evaluating and advancing new
and emerging architectures to enable artificial intelligence and machine learning applications that require
real-time analysis of large-scale data at energy-constrained environments. There are also projects on
models for quantum and classical networks for secure communications, where measurable progress has
been made toward understanding the potential range of uses of quantum computing and networking
through modeling and simulation.
Data Intensive Sciences
The data intensive sciences program has made progress in applied Machine Learning (ML),
neuromorphic/low-power computing, and cooperative multi-agent control using deep reinforcement
learning. The data intensive sciences overview and presentations on neuromorphic processing and
cooperative reinforcement learning were excellent. The panel congratulates the data intensive team on its
strong start in machine learning. ARL has not only hired new talent but also leveraged its existing
researchers and external collaborators. The panel recommends considering a range of low-

power/neuromorphic computing frameworks and the development of practices for leveraging the
expertise of external collaborators to enhance the research of ARL scientists.
Predictive Sciences
Current predictive sciences work at ARL focuses on developing the various “multi-“capabilities that
are required for accurate computational analysis, which includes multi-scale, multi-disciplinary, and
multi-fidelity analysis. In each of these regimes, ARL researchers have a stated goal of including relevant
verification and validation (V&V) and uncertainty quantification (UQ) capabilities in their computational
analyses. A related R&D goal is to leverage results of prior computations toward developing various
surrogate reduced-order models that retain the accuracy characteristics of the original models while
requiring significantly lower computational time. The panel observed a general improvement in the
quality of predictive sciences R&D from 2015, and in particular, noted a decreased variance in quality
across the range of efforts presented.
SCIENCES FOR MANEUVER
In general, research presentations and posters were professional, logical, content-rich, and useful.
Clear growth in knowledge content by ARL researchers and support staff was demonstrated. Significant
advances in the use of analytical and simulation tools were observed. The collaborative interactions—for
example, the Collaborative Technology Alliances (CTAs) and Collaborative Research Alliances
(CRAs)—continue to be productive. The Board noted the various director-level responses to previous
Panel recommendations. These positive responses are also reflected in the continuous improvement in
Campaign research performance.
Several research programs were observed to be outstanding. Three such research programs stand
out—research on low-ranked representation learning of action attributes (flexibility and extensibility) in
focusing on human action attributes; research on autonomous mobile information collection using a value
of information-enriched belief approach (projected functional stochastic gradient based-approach with
teams of robots); and research and simulation work on the Wingman Software Integration Laboratory,
which has a clear path to Army-relevant static and dynamic scenarios and multiple-machine and multiple-
human interactions.
Intelligence and Control
The overall technical quality of the intelligence and control effort is good, and has shown continual
improvement—particularly since the 2015-2016 assessment by the ARLTAB. The group has benefited
from the hiring of highly skilled postdoctoral researchers, some of whom are being groomed to become
full-time ARL employees. Publication in peer-reviewed journals and participation at professional
conferences has continued to grow, coupled with increasing participation in other professional activities.
Collaborations with peer communities and reputable academic groups appear to be healthy, and provide
the researchers with invaluable networking opportunities and options to leverage quality research
elsewhere. The investment in quality R&D, especially in areas less likely to be pursued by academia, has
increased the potential for impact. The connections between the individual research projects and the CTA
and CRA programs are very useful and are highly commended. While it would be a mistake to expect all
basic research to be tied into the CTAs, the CRAs provide rich sources of data and research problems, and
ready platforms for integration and testing in a research-friendly environment. The CTAs and CRAs may
naturally serve as a starting point for the benchmarks.

Machine-Human Interaction
The research generated by the machine-human interaction (MHI) group is generally of high quality,
and is focused on important MHI areas such as the development of algorithms that will allow a machine
to more effectively communicate and act as a teammate to humans. ARL’s MHI research is largely
comparable to university-led research in this area. In particular, the posters and presentations typically
contained acceptable technical content, experimental methods, presentation of data, and statistical
analysis of results. The research reflected a broad understanding of the science, and references to related
work indicated knowledge of research conducted elsewhere. The qualifications of the research teams were
well matched to the research problems and employed acceptable and often state-of-the-art equipment and
models. The research typically utilized an appropriate mix of theory and experimentation to arrive at well-
reasoned conclusions. The Wingman Software Integration Laboratory was identified as a promising
project potentially resulting in outstanding data and knowledge that could ultimately be transitioned to the
field. The project is focused on an important topic, necessary for the deployment and implementation of
human-machine teams with automated targeting. ARL has a strong set of well-qualified MHI researchers
addressing important, Army-related problems. These researchers have a unique opportunity to generate
mission-critical data from a population of specifically trained human subjects. Doing so would increase
the impact and applicability of the research while also helping the researchers better understand the needs
of the population they serve.
Perception
Overall, perception research is addressing cutting-edge problems, with meaningful and relevant
results. The group demonstrated an appropriate mix of theory, computation, and experimentation. The
group’s publications list and strategy spans the gamut from respected, application-based conference
venues to well-regarded academic conferences and publications. There is an opportunity for the group to
extend and enhance key projects to yield publications in the field’s very best journals with some
regularity. When considering the collective portfolio of researchers at individual, leading universities or
laboratories, the work achieved by ARL is comparable in scope and outcome. Together, the perception
group’s projects reflect an understanding of relevant state of the art, while demonstrating a commitment
to pursuing key open questions of Army relevance. It is clear that ARL has attracted well-qualified
research staff and provided them with excellent facilities for conducting cutting-edge research in
perception. Several of the projects were particularly well presented and showed strong promise to
transition to Army use. One such project is the online gyro calibration algorithm; another is the embodied
training project. These projects demonstrated solid understanding of the tactical ARL end point while
bringing together the proper theory or practice, as needed.
HUMAN SCIENCES
Real-World Behavior
This initiative is an ambitious and promising entry into the challenging field of the measurement and
analysis of real-world behavior (RWB). Overall, the technical quality of the work is high. In particular,
the group has worked to identify technical and theoretical gaps and to align resources to solve specific
needs. The technical quality of enabling technology and instrumentation was especially high. In general,
the group uses strong experimental techniques and appropriate modeling approaches. There has been a
continued improvement in research products, including published papers, chapters, technical reports, and
conference papers. Still, as new research areas are broached, the work would benefit by consultation with
appropriate experts.

Human Variability
The analytical abilities and techniques of the Human Variability program in general are strong. The
electroencephalogram (EEG)-related technical expertise is excellent. The source localization methods
being developed are interesting and a good approach to go beyond simple subtractive methods of analysis.
The Human Variability projects have made progress since the last review by continuing to publish
findings in the scientific literature and present findings at conferences.
Humans in Multiagent Systems
Overall, the technical quality of the work is good, and methodologies that are used to explore the
research questions are appropriate. Throughout the description of the research in this area, there were
many examples of good interdisciplinary collaboration to support broad-ranging questions among
computer scientists, cognitive scientists, and human factors psychologists and engineers, such as in the
work on trust in robotic transportation systems. The Humans in Multiagent Systems team leverages the
foundational (6.1) research of their colleagues in the Human Variability and Real World Behavior
programs to inform their applied (6.2) work as well as collaboration with operational warfighters, at Fort
Bragg, for example. The identification of the Cyber Human Integrated Modeling and Experimentation
Range Army (CHIMERA) lab as a target of opportunity for research into the human aspects of cyber
security, represents foresight into outreach and collaboration with other organizations, and it leverages
investments made elsewhere. The researchers demonstrated a clear understanding of the importance of
measures of performance and measures of effectiveness, that achievement of the former does not always
translate into achievement of the latter, and that this disconnect needs to be addressed in their research.
One area that lacks depth is qualitative research. The work on sociocultural influences in particular
seems heavily dependent upon the use of qualitative methods, but the methods used for conceptualizing
research questions and analyzing and presenting data need some additional expertise to be brought up to
the standards of similar academic research.
Another area where more depth is needed is in the area of teams research. In some areas—particularly
the work on teams in cybersecurity—the lack of deep expertise on current literature is leading to slow
progress in the research. The opinion of some ARL researchers that nothing is known about the area of
cybersecurity teams reveals a lack of knowledge of other teamwork research. Abstracting the issues of
cybersecurity teams to think in terms of complex, fast-paced decision making in the face of adversarial
pressure would reveal some relevant and useful literature to build upon.
A related area that could benefit from more expertise is in multilevel theory and analysis. Ultimately,
to translate the findings across campaigns into actionable conclusions will require integrating findings
from individual-level research into teams and higher levels of analysis. Much of the research presented
was at the individual level of analysis; the few examples of teams research made no use of information on
individual differences, which would undoubtedly affect how teams operate.
Human Cyber Performance
Because this line of research is in its infancy, it is very difficult to assess the technical quality of the
work at this time. With that said, it was clear that the researchers are beginning to come up to speed in
cybersecurity as they continue to collaborate with their technical counterparts who have a deeper
understanding of the technical components of the cyber mission. This collaboration will be essential as
the team advances its vision to develop a human science of cybersecurity.

ANALYSIS AND ASSESSMENT
The ARLTAB assessment of this campaign is different from that of most, if not all, other campaign
assessments, since the Analysis and Assessment Campaign is intended to be more of an analytically
focused, crosscutting, activity rather than being research focused. As a result, the criteria are different
from those of research-focused campaigns. Nevertheless, the work needs to have technical depth.
The quality of the technical staff and the quality of the work reviewed was generally outstanding.
Nevertheless, current analysis and assessment (A&A) efforts are falling further behind in incorporating
the complexities of the technologies and environments needed for A&A. This could be due to lack of
resources—either financial, personnel, or both. In several areas, the team is only one deep, so there is a
lack of personnel, likely driven by funding. Resources need to be made available to address the
requirements to analyze and assess complex technologies in complex environments.
The modeling work is generally of high quality, but rigorous verification and validation is not always
included in model and tool development; this is especially the case in human systems integration (HSI).
Of the core technical ballistics survivability, vulnerability, and lethality (BSVL) efforts reviewed, the
projects on underbody blast modeling and the collection of data that shows the impact of multiple hits on
armor panels were of very high technical quality and the teams were highly educated and skilled to
conduct these efforts. However, limited resources, either funding or personnel, resulted in important
effects being modeled well after this information was needed. A lack of validated models has led to the
need to carry out a very costly experimental program, illustrating that it is important to stay ahead of the
need.
Three of the Key Campaign Initiatives (KCIs), “Framework for Complex Multidomain Analysis,”
“Analysis & Assessment of Congested & Contested Operational Environments,” and “CEMA Analysis
and Assessment Methodology for Congested and Contested Environments,” have significant overlap, are
well outside the current mission space of ARL, and are so broad that the outcomes cannot be clearly seen
in the 15-year time frame. More definition and work is needed on these initiatives. The fourth KCI,
“Virtual Interactive Simulation Analysis and Assessment,” is a long-term follow-on effort from the
immersive demonstration that served as a proof-of-principle for these visualization techniques. These KCI
efforts would all benefit with near- and long-term deliverables defined.
The area of analysis and assessment is a very important area to the Army. The analyses and
assessments prepared by ARL support Army decisions at all levels of the Army. The products of these
assessments and analyses are used by the Army Evaluation Command in preparing recommendations to
Army leadership up to and at the Secretariat level. A&A is an important activity that is very
underresourced and falling further behind in meeting mission goals, which puts ARL at risk to losing this
activity to another Army organization. It is important to keep this activity in ARL to link the 6.2 tool
development with 6.6 tool application.
BSVL has historically led the way in modeling ballistic survivability, lethality, and vulnerability for
the Army, DOD, and international allied community. There is no competition for leadership in this area—
it is ARL’s mission. The movements to embrace HPC to speed computations and support the Army
community needs for A&A are commendable efforts. The emerging methodology for underbody blast and
multihit survivability analyses will be exceptional contributions to Army A&A.
The finite element modeling of underbody events on vehicles is top-notch work. The finite element
modeling team has a clear understanding of the fidelity required and is advancing state-of-the-art tools
and contributing to their validation.
The approach for the physiological experimental work combining high-speed X-ray imaging with
state-of-the-art and exploratory sensor technologies is an example of outstanding work that is leading this
field. It is the best high-speed X-ray capability that has been observed in this area.
In HSI, the human physical accommodation models and soldier performance and workload modeling
and simulation tools developed and employed by ARL are first rate and have provided the Army and
industry with an excellent capability to assess soldier integration into complex systems. Current tools,
including the Improved Performance Research Integration Tool (IMPRINT) and digital clothing and

equipment models, provide analytical capabilities that can be cost-effectively applied early in the
acquisition cycle as well as later during system development.
CROSSCUTTING RECOMMENDATIONS
Based on the 2017 reviews whose assessment is summarized in this interim report, ARLTAB offers
eight recommendations.
Recommendation 1: Upon initiation, ARL research efforts should propose a positioning plan
and schedule that includes
1. Identification of key, core, and complementary research programs and relevant
expertise;
2. A statement of ARL’s intended role in the context of this expertise (lead, follow,
support); and
3. An appraisal of the need for external and internal technical support—for example, via
external advisory boards, visiting researchers, workshops, or collaborations—and if
needed a plan to develop such support.
Recommendation 2: ARL should bring about greater understanding of available technical

expertise in critical subject areas across ARL, and leverage this expertise to build greater
synergy across campaign thrusts.
Recommendation 3: ARL should provide predictable long-term funding for multiyear projects.
Recommendation 4: To facilitate research, ARL should streamline the approval process for
conference participation and procurement of equipment.
Recommendation 5: ARL should place greater emphasis and focus on a systematic assessment
of its research. The assessment should include measureable milestones, outcomes, and metrics
for the portfolios and the projects within them. In all ARL campaigns, research efforts aimed at
developing any system should endeavor to understand, incorporate, and accommodate the
soldier within the system through the incorporation of human systems integration (HSI)
principles. HSI should include the consideration of usability, sustainability, resilience, and
survivability within the system.
Recommendation 6: ARL should develop and host a curated data repository of select Army-
relevant data, targeting domains and contexts relevant to its strategic objectives and preserving
data and contexts that may otherwise be lost. In conjunction with development of the data
repository, ARL should develop a set of Army-specific data analytics questions and sponsor
competitions to accelerate progress on ARL problems and attract new talent and expertise.
Recommendation 7: The ARL research efforts within a particular campaign should comprise
four components: (1) real-world observations (for example, surveillance, field research, and
naturalistic observations); (2) laboratory testing; (3) theoretical underpinning of the science
(for example, modeling and simulation); and (4) assessment, verification and validation, and
uncertainty quantification of the models. All research should endeavor to contribute to one or
more of these research components in such a way that each component’s findings serve to
inform the other research components. In addition, a balance should be met between the
contributions to these various components so that an overall systems appreciation is achieved.
10

ARL should further enhance the use of appropriate models to better understand the
phenomena of interest and develop technology.
Recommendation 8: To enrich the ARL open campus, ARL should consider developing an ARL
on-site open network that research staff can use to readily access research software that has not
yet received qualification for use on the internal network.
11

Introduction
This introductory chapter describes the biennial assessment process conducted by the National
Academies of Sciences, Engineering, and Medicine’s Army Research Laboratory Technical Assessment
Board (ARLTAB). It then describes the preparation and organization of the report, the assessment
criteria, and the approach taken during the report preparation.
THE BIENNIAL ASSESSMENT PROCESS
The ARLTAB is guided by the following statement of task:
An ad hoc committee to be named the Army Research Laboratory Technical Assessment Board
(ARLTAB), to be overseen by the Laboratory Assessments Board, will be appointed to continue the
function of providing biennial assessments of the scientific and technical quality of the Army Research
Laboratory (ARL). These assessments will include findings and recommendations related to the
quality of ARL’s research, development, and analysis programs. While the primary role of the
ARLTAB is to provide peer assessment, it may offer advice on related matters when requested by the
ARL Director. The ARLTAB will provide an interim assessment report at the end of Year 1 of each 2-
year assessment cycle and a final assessment report biennially. The ARLTAB will be assisted by up to
seven separately appointed panels that will focus on particular portions of the ARL program. Each
year, up to three additional panels may be appointed to assess special topics, at the request of the ARL
Director.
The charge of ARLTAB is to provide biennial assessments of the scientific and technical quality of
the Army Research Laboratory (ARL). These assessments include the development of findings and
recommendations related to the quality of ARL’s research, development, and analysis programs.
ARLTAB is charged to review the work in ARL’s science and technology (S&T) campaigns (Materials
Research, Sciences for Lethality and Protection, Information Sciences, Computational Sciences, Sciences
for Maneuver, Human Sciences, and Analysis and Assessment) but not the work of the Army Research
Office (ARO), a key element of the ARL organization that manages and supports basic research.
However, all ARLTAB panels receive reports of how the research and development (R&D) activities of
ARO and ARL are coordinated.
In addition, at the discretion of the ARL director, the ARLTAB reviews selected portions of the work
conducted by the Collaborative Technology Alliances (CTAs) and Cooperative Research Alliances
(CRAs). Although the ARLTAB’s primary role is to provide peer assessment, it may also offer advice on
related matters when requested to do so by the ARL director; such advice focuses on technical rather than
programmatic considerations. To conduct its assessments, the ARLTAB is assisted by seven National
Academies panels, each of which focuses on one of ARL’s S&T campaigns. ARLTAB’s assessments are
commissioned by ARL itself rather than by one of its parent organizations.
For this assessment, the ARLTAB consisted of eight leading scientists and engineers whose collective
experience spans the main topics within ARL’s scope. Seven panels, each of which focuses on one or
12

more of ARL’s S&T campaigns, report to the ARLTAB. Seven of the ARLTAB members serve as chairs
of these panels. The panels range in size from 10 to 16 members, whose expertise is carefully matched to
the technical fields covered by the areas that they review. Selected members of each panel attend each
annual review. In total, 91 members participated in the reviews that led to this report. All panel and
ARLTAB members participate without compensation.
The National Academies appointed the ARLTAB and panel members with an eye to assembling a
slate of experts without conflicts of interest and with balanced perspectives. The experts include current
and former executives and research staff from industrial R&D laboratories, leading academic researchers,
and staff from the Department of Energy (DOE) national laboratories and federally funded R&D centers.
Nineteen are members of the National Academy of Engineering, 4 are members of the National Academy
of Sciences, and 4 are members of the National Academy of Medicine. A number have been leaders in
relevant professional societies, and several are past members of organizations such as the Army Science
Board and the Defense Science Board. ARLTAB and its panels are supported by National Academies
staff, who interact with ARL on a continuing basis to ensure that ARLTAB and the panels receive the
information they need to carry out their assessments. ARLTAB and panel members serve for finite terms,
generally 4 to 6 years, so that viewpoints are regularly refreshed and the expertise of the ARLTAB and
panel members continues to match ARL’s activities. Biographical information on ARLTAB members
appears in Appendix B.
In 2017, the seven panels reviewed the following S&T campaigns of ARL:
 Panel on Materials Science and Engineering: Materials Research;

 Panel on Ballistics Science and Engineering: Sciences for Lethality and Protection;
 Panel on Information Science: Information Sciences;
 Panel on Computational Sciences: Computational Sciences;
 Panel on Mechanical Science and Engineering: Sciences for Maneuver;
 Panel on Human Factors Science: Human Sciences; and
 Panel on Assessment and Analysis: Analysis and Assessment.
The current interim report summarizes the findings of the ARLTAB from the seven reviews
conducted by the panels in 2017. The remainder of the S&T campaign areas will be reviewed in 2018,
and the final, biennial report, which will be written in 2018, will subsume the current interim report and
will add findings from the 2018 assessment.
PREPARATION AND ORGANIZATION OF THIS REPORT
As with the earlier reviews, this report contains ARLTAB’s judgments about the quality of ARL’s
work. (Chapters 2 through 8 focus on the individual S&T campaign areas, and Chapter 9 provides a
discussion of crosscutting issues across all of ARL.) The rest of this chapter explains the rich set of
interactions that supports those judgments.
The amount of information that is funneled to the ARLTAB, including the evaluations by the
recognized experts who make up ARLTAB’s panels, provides a solid foundation for a thorough peer
review. This review is based on a large amount of information received from ARL and on interactions
between ARL staff and the ARLTAB and its panels. Most of the information exchange occurs during the
annual meetings convened by the respective panels at the appropriate ARL sites. Both at scheduled
meetings and in less formal interactions, ARL evinces a very healthy level of information exchange and
acceptance of external comments. The assessment panels and ARLTAB engaged in many constructive
interactions with ARL staff during their annual site visits in 2017. In addition, useful collegial exchanges
took place between panel members and individual ARL investigators outside scheduled meetings as ARL
13

staff members sought clarification about panel comments or questions and drew on panel members’
contacts and sources of information.
Each panel’s review meeting lasted about 2.5 days, during which time the panel members received a
combination of overview briefings by ARL management and technical briefings by ARL staff. Prior to
the meetings, panels received extensive materials for review, including selected staff publications.
The overview briefings brought the panels up to date on the broad scope of ARL’s scientific and
technical work. This context-building step was needed because the panels are purposely composed of
people who, while experts in the technical fields covered by ARL’s S&T campaigns that they reviewed,
were not engaged in collaborative work with ARL. Technical briefings for the panels focused on R&D
goals, strategies, methodologies, and results of selected projects at the laboratory. Briefings were targeted
at coverage of a representative sample of each of ARL’s S&T campaigns over the 2-year assessment
cycle. Briefings included poster sessions that allowed direct interaction among the panelists and staff of
projects that were not covered in the briefings.1
Ample time during both the overview and the technical briefings was devoted to discussion, which
enabled panel members to pose questions and ARL staff to provide additional technical and contextual
information to clarify panel members’ understanding. The panels also devoted sufficient time to closed-
session deliberations, during which they developed findings and identified important questions or gaps in
panel understanding. Those questions or gaps were discussed during follow-up sessions with ARL staff so
that the panel was confident of the accuracy and completeness of its assessments. Panel members
continued to refine their findings, conclusions, and recommendations during written exchanges and
teleconferences among themselves after the meetings.
In addition to the insights that they gained from the panel meetings, ARLTAB members received
exposure to ARL and its staff at ARLTAB meetings each winter. The 2017 ARLTAB meetings refined
elements of the assessment process focused on ARL’s S&T campaigns, including read-ahead materials,
review agendas, and expertise required within the panels.
ASSESSMENT CRITERIA
During the assessment, the ARLTAB and its panels considered the following questions posed by the
ARL director:
 Is the scientific quality of the research of comparable technical quality to that executed in
leading federal, university, and industrial laboratories both nationally and internationally?
 Does the research program reflect a broad understanding of the underlying science and
research conducted elsewhere?
 Does the research employ the appropriate laboratory equipment and numerical models?
 Are the qualifications of the research team compatible with the research challenge?
 Are the facilities and laboratory equipment state of the art?
 Are programs crafted to employ the appropriate mix of theory, computation, and
experimentation?
To assist ARL in addressing promising technical approaches, the ARLTAB also considered the
following questions:
1
Agendas of the panel meetings can be found at the National Academies of Sciences, Engineering, and
Medicine website at http://www8.nationalacademies.org/cp/ for each respective panel.
14

 Are there especially promising projects that, with improved direction or resources, could
produce outstanding results that can be transitioned ultimately to the field?
 Are there promising outside-the-box concepts that could be pursued but are not currently
in the ARL portfolio?
Within the general framework described earlier, the ARLTAB also developed, and the panels
selectively applied, detailed assessment criteria organized in the following four categories (Appendix C
presents the complete set of assessment criteria):
1. Project goals and plans. Criteria in this category relate to the extent to which projects
address ARL strategic technical goals and are planned to effectively achieve the stated
objectives.
2. Methodology and approach. Criteria in this category address the appropriateness of the
hypotheses that drive the research, of the tools and methods applied to the collection and
analysis of data, and of the judgments about future directions of the research.
3. Capabilities and resources. Criteria in this category relate to whether current and
projected equipment, facilities, and human resources are appropriate to achieve success
of the projects.
4. Scientific community. Criteria in this category relate to cognizance of and contributions to
the scientific and technical community whose activities are relevant to the work
performed at ARL.
APPROACH TAKEN DURING REPORT PREPARATION
This report represents ARLTAB’s consensus findings and recommendations, developed through
deliberations that included consideration of the notes prepared by the panel members summarizing their
assessments. ARLTAB’s aim with this report is to provide guidance to the ARL director that will help
ARL sustain its process of continuous improvement. To that end, the ARLTAB examined its extensive
and detailed notes from the many ARLTAB panel and individual interactions with ARL during 2017.
From those notes, it distilled a shorter list of the main trends, opportunities, and challenges that merit
attention at the level of the ARL director and the management team. The ARLTAB used that list as the
basis for this report. Specific ARL projects are used to illustrate these points in the following chapters
when it is helpful to do so, but the ARLTAB did not aim to present the director with a detailed account of
interactions with bench scientists. The draft of this report was subsequently honed and reviewed
according to the National Academies’ procedures before being released.
The ARLTAB applied a largely qualitative rather than quantitative approach to the assessment. The
approach of ARLTAB and its panels relied on the experience, technical knowledge, and expertise of its
members, whose backgrounds were carefully matched to the core technical competency areas in which
ARL activities are conducted. The ARLTAB and its panels reviewed selected examples of the scientific
and technological research performed by ARL; it was not possible to review all ARL programs and
projects exhaustively. Given the necessarily nonexhaustive nature of the review process, the omission of
mention of any particular program or project should not be interpreted as a negative reflection on the
omitted program or project.
ARLTAB’s goal was to identify and report salient examples of accomplishments and opportunities
for further improvement with respect to the technical merit of ARL work and specific elements of ARL’s
resource infrastructure that are intended to support the technical work. Collectively, these highlighted
examples for each ARL S&T campaign are intended to portray an overall impression of the laboratory
while preserving useful mention of suggestions specific to projects and programs that the ARLTAB
considered to be of special note within the set of those examined.
15

REPORT CONTENT
This chapter has addressed the biennial assessment process used by the ARLTAB and its seven
panels. Chapters 2 through 8 provide detailed assessments of each of the ARL S&T campaigns reviewed
during 2017. Chapter 9 presents findings common across multiple S&T campaigns. The appendixes
provide ARL’s S&T campaigns and their mapping to the technical areas reviewed in 2017, biographical
information on the ARLTAB members, the assessment criteria used by ARLTAB and its panels, and a list
of acronyms found in the report.
16

Materials Research
The Panel on Materials Science and Engineering at the Army Research Laboratory (ARL) conducted
its review of ARL’s programs in energy-efficient electronics and photonics, materials for soldier and
platform power systems, and quantum sciences at Adelphi, Maryland, on June 14-16, 2017. This chapter
provides an evaluation of that work.
ARL’s materials sciences span the spectrum of technology maturity and address Army applications,
working from the state of the art to the art of the possible—“25 years out”—according to the ARL.
Materials research efforts and expertise are spread throughout the ARL enterprise. As the ensemble of the
materials discipline and capabilities, the area of materials sciences is one of ARL’s primary core technical
competencies. In the larger context, the mission of ARL, as the U.S. Army’s corporate laboratory, is to
discover, innovate, and transition science and technology to ensure dominant strategic land power.
ENERGY-EFFICIENT ELECTRONICS AND PHOTONICS
The research into energy-efficient electronics and photonics is intended to address the size, weight,
power, cost, and time (SWAPCT) of soldier technologies on the battlefield. The impact of advances
utilizing optical equivalents, and efficiencies realized through new radio frequency (RF) waveform and
encoding strategies, and efficiencies for directed-energy applications are envisioned as being significant
and important targets. Examples include escalation of electronic warfare technologies down to the
individual unit and soldier in a continually contested RF environment, exacerbating current power
challenges even more.
Accomplishments and Advancements
ARL has made excellent progress in enhancing its linkages, both throughout the Department of
Defense (DOD) community with the Air Force Research Laboratory (AFRL), Naval Research Laboratory
(NRL), and Defense Advanced Research Projects Agency (DARPA), and with Trusted Access Program
Office semiconductor fabrication operations with the Joint Technology Office in high-energy lasers, and
certainly with the academic and industry community through extensive research and development (R&D)
collaborations. Many of these linkages have been enhanced by the Open Campus Initiative and have had
visible, positive impact on the quality and potential for impact of the work program at ARL.
Posters were of high quality overall, on contemporary research topics, and with presenters who were
generally highly accomplished and qualified in their research area. Several of the posters provided
descriptions of Centers (modeling and nanofabrication), which provide collaborative opportunities to
drive quality of work and accelerate impact in the community at large. These Centers benefit from critical
inputs from ARL both in facilities and in unique cutting-edge Army technology challenges, and are
expected to become valuable elements of the Open Campus Initiative.
More details on some of the projects reviewed are provided in the following.
17

Circuit Design
This is an exciting program with aggressive and bold goals squarely focused on critical soldier needs.
This serves as an excellent example of how effectively a well-defined application driver can illustrate
ARL’s strong potential for high-impact contributions. The ARL team is comprised of talented and well-
informed staff, firmly plugged into evolving external commercial and government program capabilities.
ARL has also identified, and is working with, the best partners in the field.
This team has developed and executed effective strategies both to access state-of-the-art commercial
technology and to manage DOD organizational challenges associated with integrating security solutions
as noted in prior years. These strategies now include actively working pathways to leverage truly cutting-
edge technologies, including multiple DARPA programs such as the Diverse Accessible Heterogeneous
Integration (DAHI) program to integrate complementary metal-oxide-semiconductor (CMOS) with GaN
RF front-end chiplets, as well as with digital hardware implementation of next-generation soldier radio.
Ultraviolet Sources
Ultraviolet (UV) epitaxy and device development for efficient emitters and high-performance
detectors has been a long-standing program at ARL with a record of strong accomplishments. This
technology supports a range of identified objectives for soldier battlefield technology, with broader
applications in agent detection; purification and decontamination; low size, weight, and power (SWAP)
atomic clocks; and potential uses in quantum memory.
This project presented ARL advances in understanding physical processes in III-Nitride materials that
affect UV emitter performance, with challenges including injection efficiency, poor carrier transport
across hetero-interfaces, clarifying the relative impact of tunneling, scattering, polarization fields, and
various contributions to carrier localization. The work is well supported by epitaxial growth, device
processing, materials, and device modeling.
The ARL team has demonstrated creative application of the laboratory’s strong ultrafast spectroscopy
epitaxial growth capabilities, utilizing specially designed structures incorporating sequential quantum
wells with different emission wavelengths to capture time of flight in transport in pin configurations. The
program has also advanced the development of materials diagnostics relevant to understanding the unique
transport and structural issues encountered in higher power AlGaN UV emitters engineered for 10 kV-
level e-beam pumping. Time-resolved photoluminescence generally had good agreement with modeling
on short time scales, but for longer time scales the model does not properly account for localization and
the resulting strong temperature dependence. As a result, these new diagnostic techniques are revealing
valuable insights into nanoscale compositional fluctuations and local monolayer quantum well thickness
fluctuations that can contribute to recombination dynamics through localization.
Efficient High-Energy Lasers
Directed energy weapons using lasers are of great interest to the Army, but their size, weight, power,
and cost (SWAPC) remains too high for the Army platforms. ARL has begun a program to develop a
single double-clad crystalline core/crystalline clad fiber to replace the 58 glass fiber lasers at 1 kW each
that Lockheed Martin Corporation utilized in a recent 58 kW laser demonstration. The crystalline fiber
lasers have the potential for 10 times or more of the power output of glass fibers.
This is a creative program already demonstrating near-term success, with rapid and significant
progress against numerically well-defined goals. The work has been effectively managed and executed
using a combination of internal and outside resources combined with novel fabrication processes. A very
high optical conversion of 78 percent from pump diode lasers to fiber laser output has already been
demonstrated.
18

Diamond Power Radio Frequency Electronics
This new effort at ARL (fiscal year [FY] 2016 start) aims to use Raman, scanning electron
microscopy (SEM), digital image correlation (DIC), X-ray photoelectron spectroscopy, and ultra-high-
vacuum Kelvin probe to investigate optimal surface preparation and different transfer dopants. The team
has defined quantitative metrics for the transistor performance it hopes to demonstrate (15 W/mm and
frequency [fT] and maximum oscillation frequency [fmax] above 100 GHz). ARL has made good progress
over the past few months in studying the hydrogenation properties of the diamond surface. A first-pass
transistor has been demonstrated with reasonable current density (90 mA/mm) and with fT and fmax around
10 GHz. This is an area of high potential impact to several DOD mission-critical systems. It leverages
ARL’s strong metrology and material processing expertise, and there is high likelihood for device level
improvement and improved understanding of the surface transfer doping diamond mechanism.
Broadband Doherty Building Block
Power amplifiers (PAs) are needed to amplify signals for transmission in RF systems ranging from
communications to radar and electronic warfare (EW). For many of these applications it is critical to have
power amplifier high-energy efficiency (PAE) over a wide range of input/output (I/O) powers.
Unfortunately, most PAs suffer from dramatic reductions in efficiency under output power back off
(OPBO) conditions—when transmitting below peak output power. Doherty amplifiers seek to use a
transmission line network to transform the load impedance to a variable-impedance seen at the output of
the power transistor so as to draw more power from the amplifiers at low-RF output powers. Such an
impedance transforming network has an intrinsic trade-off between PAE under OPBO conditions and
signal bandwidth. ARL has demonstrated an asymmetric transformation network that improves the
operating bandwidth (1.6-3.4 GHz) while limiting the reduction in PAE at OPBO. This work represents a
solid—although incremental—engineering effort. The ARL effort has attracted DOD customer interest in
further development and customization of this PA architecture.
Semiconductor Research Nanofabrication Center
The Semiconductor Research Nanofabrication Center (SRNC) comprises a broad and well-equipped
15,000-square-foot class 100/10 facility to allow both internal and external users to build a variety of
needed prototypes and support research demonstrations. With extensive epitaxy, deposition, lithography,
processing, and characterization tools, the SRNC facility not only allows rapid progress on in-house
projects but also enhances collaboration with outside researchers, which helps to facilitate the Open
Campus Initiative. One of the difficulties of operating such a facility is maintenance, and ARL has
implemented small fees charged to most outside users to support this, as well as staffing the facility with
professionals to maintain and help the scientists use the equipment.
Integrating Energetic Materials On-Chip
The team has developed a unique technology that uses porous silicon (which can be formed directly
on-chip), as the basis of a highly energetic material. In brief, the porous silicon is first formed via an
electrochemical etch, and then in-filled with a strong oxidizer. Upon triggering, the oxidizer and silicon
react explosively. The effort at ARL started 10 years ago, and is now supported by both ARL and
customer funds. This is a project with clear Army relevance, and the team has a number of publications
and patents demonstrating their leadership in this area. The project has matured to where it currently is
19

more a 6.2 effort than a 6.1 effort, although there are still 6.1 aspects such as development of new
fabrication methodologies. This is a clear example of the kind of project ARL needs to support.
Nano-Engineered Polymer Dielectrics
The investigators propose to use electron traps to improve the breakdown strength of polymers. Initial
results indicate an increase in breakdown strength of 5 to 20 percent, but the physics behind the
improvement is not understood.
Two-Dimensional Flexible Electronics
While not addressed in the poster, a key motivation for this work is that emerging two-dimensional
(2D) materials, such as MoS2, may be more amenable to enabling low-cost, scalable printing technology
solutions for flexible electronics. These materials may also lend themselves to high-frequency, high-
performance requirements. Work at this time on this early stage project is making credible progress on
understanding the basic characteristics of interfaces on plastic films, comparisons to structures on SiO2,
and development and integration of low-resistance contacting structures into basic devices that can scale
to complex circuits.
Modeling the Modulation Transfer Function in Infrared Focal Plane Arrays
This work is a direct example of the modeling strengths resident in ARL’s new Center for
Semiconductor Modeling (CSM)—here applied to capture the impact of lateral carrier diffusion on the
modulation transfer of HgCdTe and nBn focal plane arrays. Combining finite-difference time-domain
electromagnetics with finite element method (FEM) carrier dynamic modeling, the project has
demonstrated the ability to create accurate three-dimensional (3D) models for realistic pixel structures
that would enable evaluation of design variants for ARL and its industry partners. The project will benefit
from continued strengthening of U.S. Army Communications-Electronics Research, Development and
Engineering Center’s Night Vision and Electronic Sensors Directorate and industry partner engagement.
Ultraviolet Avalanche Photodetector Research
This study aims to improve SiC-based single-photon counting avalanche photodetectors in SiC,
AlGaN/SiC and InGaN/SiC for Army-relevant deep UV and near-UV wavelength regimes not well
addressed by commercial technologies that often require bulky photomultiplier tubes. The ARL effort
spans from basic epitaxy techniques demonstrating 2D epitaxy of GaN on SiC using monolayer AlN
buffers to high-gain (M > 1000) SiC avalanche photodiodes with peak response at 226 nm to optimized
quenching circuits for system applications. This is a well-connected effort with globally leading
researchers, and the materials growth and device work are state of the art.
Efficient Erbium-Based Mid-Infrared Laser System
This project has demonstrated substantial enhancements in mid-infrared (mid-IR) laser efficiencies
using a novel combination of cascade lasing and follow-on optical amplification using the higher energy
1.6 µm photon from the Er ion cascade laser to pump a Cr:ZnSe gain medium for the lower energy 2.7
µm mid-IR photon from the cascade laser. Making efficient use of the original 1 µm pump offers promise
20

for substantial improvements in SWAP. Significant work lies ahead in addressing system complexity and
assessing efficacy and power scaling qualities.
Materials Development for Piezo Microelectromechanical Systems
This program has developed a combination of basic, mission, and customer work to advance the
development of piezo microelectromechanical system (MEMS) devices for RF systems and sensors
across a variety of applications. The device work is of high quality, both in design and processing.
Miniaturized Shape Memory Alloy Microelectromechanical Systems Actuators
This program aims to assess the scaling potential of shape memory alloy (SMA) for MEMS actuators
down to thin-film device dimensions. Reversible SMA effects were observed down to 100 nm thickness,
and SMA actuation at 2 kHz for millions of cycles has been demonstrated. The basic concepts appear
viable.
Challenges and Opportunities
Many posters presented only scientific or technological objectives without clearly articulating the key
value proposition from an Army perspective, and some junior-level presenters did not seem to know or
understand the nature or future potential of the customer funding. However, upon questioning, most
presenters were able to address relevant applications issues and customer value proposition well. In
relatively few cases, the ARL work was neither fundamentally state of the art nor strongly application
focused, reducing potential for high impact. Management guidance and vigilance to address the
risk/impact balance is important.
Specific opportunities and challenges for some of the projects reviewed are discussed in the
following.
Circuit Design
Further discussions in the breakout sessions made it clear that the effort at ARL is quite deep and
comprehensive, including all the circuit design using the various platform development kits. The
aggressive design for a digitally sampled passive mixer front end, developed in the context of highly
effective academic collaborations, serves as a great target for DAHI technology. Both the 2.5D field
programmable gate array (FPGA) integration and 3D DARPA chip stacking provide pathways to scaling
to higher CMOS nodes in the future. This is a well-conceived program with performance targets that
cannot be readily achieved in CMOS due to need for high Internet protocol 3 (IP3) intercept.
Ultraviolet Sources
Introductory comments illustrating the breadth and history of ARL work in this area would have been
helpful to provide a deeper context for this work, and more discussion of how the impact of the work is
supported through device modeling and design would be useful to see.
21

Efficient High-Energy Lasers
Since the number of pump laser diodes for the glass fiber and double-clad single crystal fiber may be
comparable for both approaches, it will be prudent to develop a detailed SWAPC comparison between
these two approaches to quantify the benefits, including trade-offs in required pump-module size, number,
and cost; size and cost of multiplexers; and cost and number of wavelength multiplexed sources that can
be supported in the achievable gain bandwidths. Such an analysis can help to quantify the expected laser
system improvement and better inform the direction for the research.
Diamond Power Radio Frequency Electronics
Wide-bandgap (WBG) semiconductors have been important for both high-power and high-frequency
applications from radar to power electronics. While GaN and SiC electronics have already had significant
commercial and military impact, diamond is potentially the highest performance WBG material. Until
recently, it has been difficult to effectively dope diamond so as to generate the mobile electrical charge
needed for electronics. Recently, surface transfer doping—where a transfer dopant is introduced as a thin
film on the surface rather than impurity atoms distributed within the bulk—has been demonstrated as an
effective doping mechanism by a group in Israel. There remain several open questions regarding the
surface and interfacial properties required for optimal doping.
Nonlinear Plasmonic Transport and Plasmonic Metamaterials
At high mobility, 2D electron gases in semiconductors can exhibit plasma hydrodynamic behavior—
most notably the existence of a high-frequency plasma instability. ARL has an exploratory effort that is
considering potential applications of these plasma hydrodynamic phenomena. The researchers are solving
the coupled balance equations (semiconductor analogue of the Navier-Stokes equations). The group is
exploring a wide range of different phenomena from noise, to the boundaries between various transport
regimes, to interactions with THz electromagnetic waves. As this is a small exploratory project, the group
would benefit greatly from validating their model against existing experimental data in the literature, and
using their model to explore a single promising application (for example, tunable THz detection or
efficient THz generation) to design and rigorously model a proposed device design. With sufficient
benchmarking against existing experiments and a compelling design, this work is likely to draw interest
from several DOD customers. This is an off-center project with a potential for useful insight and real
application.
Center for Semiconductor Modeling of Materials and Devices
A collaborative project stemming from ARL’s Open Campus Initiative, the Center for Semiconductor
Modeling of Materials and Devices aims to leverage shared advances in concepts and implementation of
multiscale modeling methodologies together with ARL’s powerful high-performance computing center.
In addition to bringing together community scientific and numerical modeling expertise from academic,
government, and industry leaders, the Center for Semiconductor Modeling of Materials and Devices will
benefit from a host of highly relevant Army materials and device problems of interest. Workshops are
already under way with diverse participation and encouraging collaborations developing. Focus will be
required to achieve outcomes greater than the sum of the parts.
22

Metallocene Membranes
The team is investigating metallocene-based polymer membrane systems with suggested Army
applications including coatings, thin-film electronics, sensors, and self-healing structures. The specific
scientific or technological objectives for this project are rather unclear, and the poster did not articulate
why metallocene membranes will offer advantages over other membrane systems. While there are
interesting graphics on the poster illustrating how metallocene membranes could form into
supramolecular structures with interesting morphologies, so far it is not clear that such supramolecular
structures will form. The investigators need to define the key scientific or technological objectives, and
then focus their efforts on meeting those objectives.
Nano-Engineered Polymer Dielectrics
The investigators plan a program of work to better understand if the improvement in breakdown is
due to the electron traps, and also to start forming their own polymer dielectric films so they can better
control the dopant concentration and distribution in the polymer film. The investigators are rightly
concerned however that making their own films will be challenging and industry collaborations with
experienced commercial polymer vendors may help address this. Also, the team would benefit from clear
performance metrics and associated impact to define success and guide future investments and
benchmarks for this 6.2 project.
Ultraviolet Avalanche Photodetector Research
The authors are addressing some important technical issues, but concerns remain about the influence
of misfit stresses on device behavior.
Materials Development for Piezo Microelectromechanical Systems
The program could benefit from a stronger materials effort. There was no information on how the
investigators will evaluate the basic mechanisms of irradiation damage in lead zirconate titanate (PZT)
films, and the need for deeper capabilities in the structural evaluation of materials appears to be of
paramount importance at ARL.
Miniaturized Shape Memory Alloy Microelectromechanical Systems Actuators
Gaining a full understanding of the NiTi material and possible phase transformations is still at an
early stage. ARL may not have in-house capability to address all the technical issues in doing this,
suggesting that external collaborations may be beneficial here.
MATERIALS FOR SOLDIER AND PLATFORM POWER SYSTEMS
The research programs of materials for soldier and platform power systems (MS&PP) are motivated
by soldier battlefield power needs both currently and in the future. The research supports the tactical unit
energy independence (TUEI) essential research area (ERA), with focus on unburdening the soldier by
making power lightweight, providing power on-site, and diminishing power needs, all essential enabling
factors in supporting soldier welfare and effectiveness.
23

The MS&PP portfolio presented for review included a large emphasis on electrical energy storage
and conversion, for example, with batteries, fuel cells and capacitors, pyroelectric and radioisotope power
sources, and others. Battery work, particularly focused on novel electrolytes, is a long-standing, high-
quality research topic at ARL and is showing substantial returns. The work on high-voltage aqueous
electrolytes could be revolutionary for battery technology for the Army and elsewhere. Work on
radioisotope-based power sources is also noteworthy, having progressed very rapidly from concept to
implementation. This work has significant upward potential. Fuel-cell and other electrochemical energy
conversion work is of high quality, but could benefit from finer definition of how it is distinguished from
other work being pursued worldwide. Combustion catalysis work has clear relevance to Army needs and
has potential to reduce mass and volume of energy carried into the field.
Work on pyroelectric materials for energy harvesting has progressed rapidly in a short time,
particularly on the modeling front, for which the team is praised. Work on phase-change materials for
thermal management is of high quality with excellent synergy between experimental and modeling work,
and is foundational to leveraging the rapid developments in WBG power semiconductors. The work is
meeting the unique army needs for heat dissipation and shielding, and showing promise for use in pulsed
power and directed energy applications.
Alkaline and Bipolar Membranes for Fuel Cell Systems
Army work on alkaline exchange membrane (AEM) fuel cells has adapted to include a major focus
on bipolar hybrid membranes with an alkaline AEM bonded to an acidic proton-exchange membrane
(PEM). The overall goal is to support Army power needs in the field using transportable energy-dense
fuels that are liquids or solids—for example, alcohols, or other energy-rich materials.
The team has done excellent work modeling continuum behavior associated with ion and water
transport in membranes by using an overall cell voltage balance with membrane transport, pH variation,
and simple reaction kinetics. These serve to address mechanical stresses that are caused by water
generation at the AEM/PEM interface. The mechanical modeling of stresses at the interface is excellent
and important. Overall, the integration with outside groups working on AEM devices is improved, with
connections to Andrew Herring at CSM and Paul Kohl at Georgia Tech being particularly noteworthy.
Long-Life Isotope Power Sources
Beta-photovoltaics potentially offer new power cell design alternatives and performance regimes that
may be advantageous for soldier applications relative to more thoroughly studied beta-voltaic power
sources. ARL’s experimentally demonstrated power levels appear to be approaching useful performance
targets even at an early stage, and the program is benefiting from quantitative 10 mW power generation as
an intermediate goal.
In Situ Spectroscopic Study of Catalytic Oxidation of Propane on Platinum
The goal of this work, to reduce the weight/volume of logistics fuel and combustor needed to provide
energy for Army missions, is well aligned with the ARL mission. Evidence of platinum oxide formation
during catalytic fuel combustion is reported. The team shows broad understanding of the key issues in
their research and has produced several high-quality publications. The team has appropriate equipment
and facilities, and is using appropriate models. The research includes both experimental and
24

computational modeling aspects; the density functional theory (DFT) modeling with Delaware is a
strength.
Aqueous Batteries and Beyond
The aqueous battery research on Li-ion batteries at ARL is of very high scientific quality, and is
receiving international recognition for its highly innovative discoveries, carried out by a very small group.
This is an “especially promising project” that merits strong internal support and that will attract strong
external recognition. The development of the WiSE/polymer gel is an “outside the box” approach that
could revolutionize the component materials and fabrication of future Li-ion batteries. The participants
have a deep understanding of related work going on elsewhere, and are in touch with the best battery
research laboratories throughout the world. They are keenly aware of international research advances
where there is currently very high quality work being reported elsewhere at a high rate of discovery. The
ARL group has identified a niche based on clearly stated and justified Army needs, and is pursuing it with
high effectiveness. Its work applies broadly to the Li-ion battery field and is therefore likely to have a
very high impact. The effort to date has generated the clear need to pursue applications at the 6.2 level, as
well as additional fundamental studies at the 6.1 level.
The unique strength of the group is its combination of physical intuition (which guides it in exploring
innovative materials for lithium battery systems) and computational abilities (which provide the means of
exploring material behavior at the atomistic scale). The development of the WiSE/polymer gel that
enables the presence of small amounts of water that enables higher voltage operation without associated
stability aspects is the key discovery. The concept has been demonstrated in the laboratory in coin-size
cells that have operated up to 4000 cycles without degradation. The mix of theory, computation, and
experiment is exemplary, and deserves recognition as a model for assembling multidisciplinary teams that
explore with skill, and initiate research and engineering connections within the laboratory as needed to
make rapid advances.
The laboratory equipment is excellent, and includes synthesis and characterization methods, small-
scale cell fabrication and testing equipment, and high-performance computing. The research team has an
excellent portfolio of skills, qualifications, and, especially, attitudes toward the different types of
problems that arise, which span from fundamental science to engineering design and development.
Pyroelectric Energy Conversion
This project focused on development of pyroelectric PZT, PLZT, and HfZrO2 materials, and has a
balanced combination of experimental and modeling work, with modeling being completed with the
University of Connecticut. The background and relevance were well presented with a good “survey of the
landscape” of how the work relates to other research outside ARL. The goal is to increase the efficiency
of conversion of heat to electricity. The team consists of a mixture of ARL investigators, students, and
outside collaborators. The experimental characterization of the materials being done at ARL is
commendable. Three journal papers have been published in high-impact journals, and the team appears to
be heading in the direction of further unique results. The lead on this project appears to have a clear vision
of where this project is going, and it has potential paths forward for future efforts.
Phase Change Materials for Electronics Thermal Management
This project examines metal-based phase-change materials for transient thermal management in
power semiconductor devices, to avoid overheating and possibly to include thermal signature reduction.
The focus on new WBG power devices is essential to capitalize on the heavy investments the DOD has
25

provided over the last several decades, and to help direct further device developments toward Army
needs. The combined experimental/modeling approach is excellent and is lending insight into how new
thermal management materials could be used in devices. The speaker was a very energetic researcher who
understood the project goals quite well. The work was based on a well-cited body of knowledge, and had
good collaboration with five outside academic and federal laboratory groups. The speaker felt that current
materials were sufficient to satisfy project goals and new material development was not necessary for this
project. The team is well qualified, with broad understanding of the issues and trade-offs involved in the
research. The experimental equipment, facilities, and access to modeling expertise have been used with
skill to fabricate chips for experimental investigations, and to predict fast-transient behavior associated
with reliability studies.
Modeling High-Voltage Wide Bandgap Insulated Gate Bipolar Transistors
This work is leading and essential. The work uses advanced modeling at the device structures level to
predict charge distribution during steady state and, particularly, transient operation, and is well supported
with experimental verification, requiring capabilities nearly unique to ARL. Partnerships with the device
manufactures—for example, Wolfspeed—are well developed, and the work is well grounded on cited
works. The research team is well qualified, and research results are well documented through
publications.
Modeling of High-Voltage Wide Bandgap Gate Turn-off Thyristor
The DOD has invested heavily in WBG power semiconductor devices for over two decades. The
opportunities for high voltage, high power with SiC is well recognized. However, partnerships with
device manufacturers, such as Wolfspeed, and further investment are needed to evolve very high power
devices. The gate turn-off thyristor is being evaluated as one possible device. The device is targeted for
high pulse power discharge applications, with focus on “turn-on” characteristics. Advanced modeling is
developed for such transient characterization, and is impressively verified through difficult testing for
operation into destruction. Weaknesses in device packaging for pulse power have been identified. Typical
thyristors for high power use “press-pack” technology rather than wire bonds, as can be concluded by
testing to date. A greater focus on packaging may be appropriate now that a datum is established.
The work is very advanced and essential with high returns in areas as directed energy, with possible
dual-use application in terrestrial power systems. The team is strong and takes advantage of unique
pulsed-power capabilities of ARL, combined with semiconductor modeling. Because of the complexity of
the device physics, the mix of simulation and experimentation is suitable and expected.
Additive Manufacturing for High-Temperature High-Voltage Power Packaging
This work is exploratory in developing new thermal structures that can be created by additive
manufacturing (AM). This gives ARL a beginning in the AM technology crossed-coupled with enabling
new thermal management technology to be used in power electronics. The work applies sprayed copper
for electrical and thermal conduction. The project applied current manufacturing, measurement, and
modeling techniques, which were not innovative, but were appropriate. The project was clear in stating
and accomplishing its goals and moving the technology forward.
26

(Al)GaN High-Power Electronic Devices
This device program targets vertical devices on native AlN substrates, with early work assessing
commercially supplied AlGaN films and planar processed devices. Atomic force microscopy and X-ray
diffraction with reciprocal strain mapping are employed for morphology and crystalline quality. The
device effort is adequate and in a preliminary stage.
Infrared Plasmonic Effects on Electrochemistry
This project proposes use of IR irradiation of catalytic metal electrodes to cause removal of poisons—
for example, chemisorbed CO—to produce higher activity catalysts. Preliminary IR
spectroelectrochemical studies on formic acid oxidation at platinum revealed the due pathway reaction
mechanisms in that at a low electrode potential, the reaction proceeds primarily via adsorbed formate as
the reaction intermediates; while at a high potential, through adsorbed CO, and studies on the hydrogen
evolution reaction (HER) reveal the effect of CO poisoning on the reaction rate. A broad understanding of
electrocatalysis and the importance of eliminating poisoning is demonstrated. The project includes a good
mix of theory and experiment, with the theory coming via collaboration with Boston University. The team
is well matched to the challenge of studying IR irradiation effects on electrode reactions.
Radioisotope Power Sources
This poster describes work on the use of tritium sources in beta-voltaic power sources, in which beta
emission from tritium excites electron-hole pairs in semiconductor p/n junctions located near the source,
producing a steady-state electrical current across a potential difference. The project started as a recent
seedling and has produced a 100 µW source, with plans to create a 10 mW source in the near future. The
latter could provide sufficient output to power a small remote sensor for the lifetime of the radioisotope
source, which for tritium is approximately a 12- year half-life. Work in progress seeks to improve output
using microstructured columnar p/n junctions to produce and capture more e/h pairs per beta particle. The
project is unique and seems well matched to Army needs. It includes a healthy balance of
theory/computation and experiment. Connections with prospective private-sector partners are in place that
could produce field-deployable power sources in a reasonable time frame.
Electrolytes for Next-Generation Lithium Batteries
The search for improved electrolytes for Li-ion batteries is based on expanding the voltage window of
operation, with special attention to oxidative stability of electrolytes at cathodes, while achieving
temperature and safety requirements. This area of research is broadly pursued worldwide, and in a wide
variety of defense and commercial entities. In this small 6.1 project, ARL is pursuing a niche for which it
has the clearly relevant skill and background knowledge. The team has a clear understanding of Army
needs vis-à-vis other major players with strong programs adjacent to the ARL research space; these
include Navy, SAFT, and auto manufacturers.
The unique concepts, published elsewhere but reinterpreted for this purpose, include new solvents and
additives that influence the solid-electrolyte interphase film at the anode as well as the cathode.
Electrolytes prepared from organic sulfone solvents containing lithium fluorosulfonimide salts are of
particular interest but reference is made to nitrile and fluorinated solvents with and without additives, and
solvents having high salt concentration. These directions are all sensible and worth exploring. The
critically important laboratory equipment and facilities for carrying out this work are in place. The skill
set of the personnel is well matched to the project. The team has many excellent collaborative
27

arrangements in place, including with multiple Department of Energy (DOE) laboratories and some
private-sector partners. This is excellent research that needs to continue.
There is a good match of experimental work with computational work based on molecular orbital
theory to explore electrolyte stability. Patents are emerging for the sulfone electrolyte chemistry.
Li-Ion Hybrid Capacitors for Embedded Power
Innovation of new supercapacitors configurations for rechargeable high-temperature and long-life

power applications are investigated by combining the best components of two systems from existing
high-energy and high-power systems. The work is supported by industrial collaboration for cell
fabrication and by university collaboration for analysis tools.
The scientific and technological quality of this small-scale project is high, and discussion during
poster presentations indicated a clear understanding of the background literature and underlying
principles. The collaborative group has the equipment and research facilities needed for this work.
The scientific focus is on interface stability and electrolyte breakdown. These issues will be addressed
based on foundation work published at ARL and elsewhere in a two-prong approach. One prong involves
the use of improved electrolyte additives to reduce oxidation of the activated carbon electrode. Industrial
partners will assist in procuring additive materials, and establishing purity metrics. The second prong
consists of creating a hybrid approach by combining Li-doped hard carbon electrodes with porous
activated carbon electrodes. Experimental data based on these concepts have demonstrated improved
performance for a range of additives and electrode structures, and have achieved performance metrics that
approach the design goals required for the application.
Center for Research in Extreme Batteries
The scientific quality of the battery group at ARL is very high and is internationally recognized. The
Center for Research in Extreme Batteries (CREB) provides a venue for a new level of collaboration that
will address gaps that are not addressed elsewhere in military/government/commercial sectors. The focus
on “extreme” batteries is central to Army demands for long life, high power, wide operating temperature,
and low maintenance. The growth of the battery field is very rapid worldwide, and the demand for
experienced battery researchers far exceeds the number currently in the field. The CREB concept serves
both to advance understanding of batteries in extreme environments as well as to promote through
collaboration the expansion of skilled researchers in the field. The underlying science and engineering
skills elsewhere have been recognized and are embedded in the initial steps of constructing the CREB.
The range of equipment associated with the collaborators will enable a wide range of initial projects.
Mechanisms for Isomer Energy Release: Rhenium Campaign and Nuclear Excitation by Electron
Capture
This project examined a very clever energy storage and release idea via stimulated nuclear excitation
(nuclear excitation by electron capture) to produce metastable nuclear excited states that could
subsequently be made to release their energy upon a second stimulation. This highly unusual process has
the potential to allow for release of large amounts of stored energy on demand, which could be of
significant benefit to the Army. The project is in very early stages with many aspects as yet unsolved, but
it provides the potential for high payoff. The team is knowledgeable and demonstrates deep
understanding. The project appears to be adequately resourced, though facilities were not seen and some,
possibly much, of the experimental work was done off site. It is premature to suggest allocating
28

significant new resources to the project but it does provide a high-risk high-payoff route to energy storage
that could be of high value to the Army.
This project has already produced some high-quality (very publishable) data on excited nuclear states
and has demonstrated the existence of an excited long-lived state. Support of a small number of such
outside-the-box projects is encouraged.
Design and Development of a Combustion-Based Portable Power Platform
This project demonstrates and examines a technology involving catalytic combustion of complex
fuel—for example, JP-8 or jet propellant 8 (fuel)—for maximizing thermal-to-energy conversion. It uses
mostly existing materials, computational resources, and approaches to guide small-scale research reactors
toward thermal-to-energy conversion at high efficiency. The research team is well qualified with adequate
resources. The project includes a good balance of theory and experiments. The project had well-defined
goals of increasing energy density for dismounted soldiers. These goals were met by the small-scale
research reactors developed by the team.
The experimental combustion reactor is configured to facilitate comparison between physical
experiments and numerical computations. The fluid dynamics are modeled with an object-oriented finite
volume approach that can be modified by code users. The parameters associated with about 2000
reactions are available. Results to date serve to demonstrate that numerical experiments can be performed
and compared with observations from physical experiments. The combined approach is structured with
the goal that it may be modified as new experimental results become available. The results to date have
identified the need for better high-temperature emissivity data. Future studies with this approach can be
used to guide scale-up and associated choice of materials and manufacturing methods, to optimize flow
geometry and catalytic performance to achieve reduced heat loss and improved efficiency.
The plan is to create an open-source code so others beyond the creator can make adjustments. With
these improved design tools and insights, the project may be ready for transition to a technology
development team.
The review of the combustion catalysis work was too simplified to show the strong impact, and
would benefit from substantial clarification of the linkages to other areas being pursued—for example,
JP-8 production and use. A clearer link between catalyzed combustion and electrical energy conversion—
for example, using thermoelectric materials—would help one see more clearly the systems-level benefits
of this approach to providing soldiers with power.
A suggestion is that future presentations provide modestly more detail in how the subsequent selected
talks link with the broader division activities, and identify research expectations.
following.
Alkaline and Bipolar Membranes for Fuel-Cell Systems
The bipolar membrane configuration is widely known in electrolysis cells, but less studied for
galvanic cells such as fuel cells. It offers the advantage of acidic operation on one side and alkaline
operation on the other side, which can allow for separate optimization of catalysts for anode and cathode,
and for alternative water management schemes. A key limitation of such cells is the AEM/PEM interface,
which can be mechanically unstable and could provide undesired high interfacial impedance.
29

The project has not pursued much experimental work addressing the structure or properties of the
various components that compose the AEM/PEM interface, which is critical to the operation of bipolar
cells. More work is needed here focused on this interface, with attention to include quantitative metrics
and accelerated testing protocols.
Project work in this area is somewhat hampered by a lack of clarity on choice of fuel. Fuel-cell
materials development work follows quite different paths for different fuels and different scales of
operation. For example, it is not clear that developing fuel cells using hydrogen as fuel will be helpful for
creating fuel cells using methanol as fuel. This project has pursued little work on methanol crossover,
which is a deficiency since crossover is perhaps the leading cause of inefficiency in direct methanol fuel
cells. The project could be managed more easily if there was clarity on the fuel being targeted.
The team would benefit from a clear description of their “niche”—for example, how they differ from
the many other groups worldwide that are working on AEM-based electrochemical devices. The team’s
niche may be the bipolar membrane approach, and if this is the case they could work toward
demonstrating clear advantages of that configuration relative to conventional PEM or AEM approaches,
especially for Army applications. Doing so would establish ARL as a significant player in this field.
Long-Life Isotope Power Sources
More effort toward detailed analysis of phosphor photon yield optimization and photon collection
efficiency will be essential to understanding and achieving ultimate performance potential.
In Situ Spectroscopic Study of Catalytic Oxidation of Propane on Platinum
It was not clear whether the platinum oxide formation would be a problem at the temperatures at
which the cell would operate. Potential problems with sulfur in logistics fuel are likely and were not
addressed. A specific examination of sulfur effects that are likely to be important in catalytic combustion
technologies is needed.
Aqueous Batteries and Beyond
Continued fundamental work is justified for the search for alternative salts, for high-throughput
computational work to identify regimes of stability. The concept could be applied to other metal systems,
such as Na, that would reduce cost. Continued applied work is needed to scale up the cells, and to explore
new methods of fabrication with the participation of industrial partners. The computational platform could
be expanded so that others can use it and modify it for numerical studies on additional design and
engineering. Important aspects of this research include high-speed computations suitable for optimization,
identification of the most sensitive parameters, uncertainty quantification (UQ), and tracking of errors
bars that arise from experimental as well as numerical sources associated with the prediction of behavior.
Phase Change Materials for Electronics Thermal Management
A suggestion on further direction is to provide more guidance to material formulators in addressing

higher temperature capabilities, and to provide a benchmark with non-phase-change materials. This
project crosscuts many technologies, such as electronic packaging, electronic materials, power
electronics, and thermal systems, which is a technical strength. However, outside recognition of the work
will be difficult since it does not align with a singular identifiable research area, such as batteries. The
general difficulty in recognition of multidisciplinary results also applies to ARL’s broad spectrum of
30

contributions. A suggestion is that ARL consider hosting a yearly “summit” that provides an open forum
for display of its research endeavors, and may tie well with an “open campus” open house. This also
aligns with the reinitiation of the Interagency Advanced Power Group.
Modeling High-Voltage Wide Bandgap Insulated Gate Bipolar Transistors
The DOD has invested heavily in wide bandgap (WBG) power semiconductor devices for over two
decades. The opportunities for high voltage with SiC are well recognized. Commercial market forces are
driving lower voltage, for example, 600 V to 1700V, WBG devices, particularly metal-oxide
semiconductor field-effect transistors (MOSFETs). However, DOD investment is needed in continued
manufacturing development of very high voltage, high-power insulated gate bipolar transistor devices,
which are recognized to best perform over MOSFETs in the 20 kV to 27 kV range. The Army has need
for better devices in areas such as high-density Marx Generators.
Additive Manufacturing for High-Temperature High-Voltage Power Packaging
A key technology development with new materials and processes is additive manufacturing (AM).
Application of AM to power electronics is in its infancy, and is an area that needs aggressive pursuit. A
growing proportion of electrical power used in the battlefield is processed through power electronics.
Concurrently, AM has been identified by the Army as a critical need for use in “manufacturing and
remanufacturing at point of need” in the battlefield. As such, developing AM processes in creating power
electronics modules provides an early technology development to support a critical Army need. A
suggestion going forward is to also consider long-term integration of sprayed ceramic into such structures
as a replacement to ceramic plate to further enhance thermal performance.
(Al)GaN High-Power Electronic Devices
The material effort needs to be strengthened and the coupling between device and materials teams
could be stronger. This will become very important when devices may not work as designed, and the team
will benefit from deeper familiarity with the published literature in this field.
Infrared Plasmonic Effects on Electrochemistry
The anticipated effects of IR irradiation on electrocatalytic activity are still somewhat speculative,
since IR plasmonic experiments have not yet been done on catalytic electrodes but are planned following
fabrication of IR-plasmonic-active electrodes by microfabrication. The team might benefit from
collaboration with condensed-matter physicists, who could provide a more detailed description of the
effects expected from IR irradiation of electrodes. It is not fully clear which groups if any the team is
collaborating with at ARL on this project. The team would benefit from a close collaboration with other
electrochemistry experts at ARL focusing on electrocatalysis for energy conversion.
Electrolytes for Next-Generation Lithium Batteries
Consideration of a wider range of additives, electrolyte molecules, and temperature is needed to guide
discovery of more stable multiple salt/solvent systems, and to understand how anions serve to disrupt
water and shield electrodes from degradation. Such exploratory studies also represent a large
31

computational demand that exceeds the availability of current personnel within the Materials Research
Campaign. While there are interactions with computing personnel elsewhere at Adelphi and Aberdeen,
the direct day-to-day contact between experimentalists and computational people could be improved.
Li-Ion Hybrid Capacitors for Embedded Power
Work on improved lifetime is a clear focus for future work. Understanding of role of additives at the
carbon-electrolyte interface in the activated carbon electrode is clearly warranted, although that represents
a difficult experimental challenge owing to the hidden difficult-to-access nature of the interface. Auxiliary
experiments in a simpler system than the entire system may be considered in order to develop improved
intuition to guide system improvements in the components of the system. The molecular structure of
additives at the solid-electrolyte interface represents an enormous field of research with applications in
many disciplines. The likelihood of drawing significant inspiration from seemingly distant applications is
therefore high.
Center for Research in Extreme Batteries
The Center for Research in Extreme Batteries (CREB) is meant to bring together industry leaders that
will contribute to the research work of the Center via membership fees to the Center. There are other
large-scale collaborations in the battery field—for example, supported by other government (DOD, DOE,
National Science Foundation [NSF], Joint Center for Energy Storage Research) and commercial
collaborations—from which examples of excellent management structure and practice could be beneficial
to the emerging CREB program. The linkages created by CREB are on the critical path for achievements
that would not be possible otherwise.
The communication between collaborators is critically important in broadly configured programs
such as this. Once the initial stages of the launch are completed, it will therefore be important to review
the organizational structure to identify opportunities for sustained communication among partners. For
example, the role of postdoctoral researchers is often critical to facile flow of information between
collaborating units through seminars on common scientific interests and training sessions on shared
equipment.
The speed of reaching cross-institutional agreements is important in such cooperative research and
development agreement (CRADA)-like interactions. For the CREB, the headquarters resides in
University of Maryland, but it took a very long time to get contracts in place. Meanwhile the focus on
industry may move on, and opportunities are thus lost. The sharing of experience among early adapters of
such programs represents a critical path issue.
Gas Removal and Mathematical Modeling for Operating Thermal Batteries
This project is long-standing and requires some new concepts in order to move it forward. The
problem is that the investment in this project continues not to resolve the issue of hydrogen removal from
batteries. Apparently, the problem is a result of very complex chemistry that is difficult to untangle, and
the investigator mentioned having a difficult time finding collaborators that can support the effort. It is
perhaps time to reconsider where this effort is going.
32

QUANTUM SCIENCES
Quantum sciences is a new program area of high scientific quality and is well aligned with the long-
anticipated that quantum sciences will provide game-changing capabilities for command, control,
communications, computers, intelligence, surveillance, and reconnaissance (C4ISR) for the Army of the
future.
In just a few years, the quantum sciences program has attracted outstanding investigators, driven in
part by their membership with the University of Maryland and the National Institute of Standards and
Technology in the Joint Quantum Institute. ARL has done an outstanding job of bringing in a strong team
of a well-established, mid-career leader with extensive knowledge of, service laboratory experience with,
and connections to military systems and needs, and a second, well-established academic who is a
recognized quantum sciences leader. In the past two years under their leadership, an excellent research
team of six experimentalists and one theoretician has been established.
In addition, ARL has built impressive research facilities with a focus on three areas—well-known Rd
atom traps to provide an experimental facility to investigate new ideas in quantum information, entangled
states and their deterministic generation, detection, and coupling; investigation and characterization of
related systems of single atoms in a solid-state host that could be operational at room temperature and be
far more easily field deployable; and investigation of newly discovered native defect systems in WBG
semiconductors: n-vacancy in diamond and the C-Si vacancy pair in SiC.
These are attractive candidate systems because they have in-house materials growth facilities to
produce host materials with unique doping and layered structures and they are photonically active in the
wavelength region of long-distance communications fibers. ARL has built up excellent characterization
facilities to determine coherence times and manipulation of quantum states in these candidate materials
systems.
The quantum sciences program is an outstanding example of vision for a future Army need—defining
specific areas that are Army unique, are not well covered by universities or national laboratories, and are
hiring recognized leadership and creating a well-funded, exciting program that has attracted an
outstanding group of young scientists and postdoctoral researchers.
Hybrid Quantum Systems for Networking and Sensing
This presentation provided an excellent description of specific projects on developing a quantum

repeater and ultra-sensitive quantum sensors. The focus is on novel materials for photonic-based sources
and detectors. The laboratory for experimental work is impressive and provides excellent facilities for
growth of novel materials and experimental characterization of new quantum defect materials and
quantum sensors. The nitrogen defect in diamond is the most widely studied solid-state quantum
structure, but it is difficult to scale up. ARL is establishing in-house materials growth facilities for the
new VSi-VC divacancy in SiC. This is a particularly attractive system, as both single-isotope Si and C are
reasonably available and this could produce a quantum state with much longer lifetime and possibly
greater ability to set and read the state variable. This is an area in which ARL is truly leading the field
and, if successful, would be a real breakthrough for a broad range of quantum-based sensors and
communications.
33

Divacancy Modeling in Silicon Carbide
This is a strong theoretical project on basic study of the electronic structure of both the ground and
excited states of the VSi-VC divacancy in SiC. ARL has excellent supercomputer resources for this effort.
In addition, ARL is establishing hot wall chemical vapor deposition growth facilities in which to prepare
unique SiC materials and has optical characterization facilities to thoroughly characterize this quantum
system.
Rare-Earth Solids for Long-Lived Quantum Memories
The principal investigator (PI) and postdoctoral researcher have set up a very impressive experimental
characterization facility to investigate different rare-earth atoms incorporated into a variety of solid-state
hosts for both quantum memories and quantum information processing. Current focus is developing
optical addressing and readout of known rare-earth crystal systems with plans to expand search for doped
materials with longer spin-state coherence times. The researchers are also well coupled into the
theoretical effort in identifying quantum states with better external perturbations and coupled interactions
in the excitation in an ensemble system with disorder.
Entanglement Swapping of Two Arbitrarily Degraded Entangled States
The goal of this effort is to characterize the reach, rates, and entanglement quality of entanglement
distribution networks in order to design optimal quantum networks for different functionalities. This is a
project of high scientific quality and is well aligned with the long-term goals of ARL’s quantum science
program to create, manipulate, and use quantum entanglement over long distances for quantum networks.
The effort has resulted in publications in the peer-reviewed literature and in conference proceedings.
Finally, the scientific strategy for future work has been clearly articulated.
Exploring Unique Electronic States at Topological Insulator-Superconductor Interfaces
This project examined PbSnTe Josephson junctions as semiconductors and topological materials.
PbSnTe exhibited proximity-induced superconductivity for the first time, although this effect has been
seen in other materials. The team demonstrated broad understanding with skills well matched to the
project challenges. The project is mostly experimental with little theory or modeling, and could possibly
benefit from some application of theory. Proximity effects on electronic behavior may have application in
quantum computing and related devices, so this work provides materials support for the discovery-based
effort on quantum devices. Most likely a high-impact publication will result from this research.
This is definitely a hot field, with a recent high-profile announcement of discovery of the first
Majorana Fermion. There are good potential Army applications if ARL can demonstrate the novel quasi-
particle physics that shows Qubit immunity to decoherence in such a quantum state. ARL has a good
theoretical effort and a unique molecular beam expitaxy (MBE) growth facility to prepare near perfect
planar superconductor/topological material interfaces required to observe a Majorana Fermion, and has
observed the first proximity-induced superconductivity in a superconductor/PbSnTe topological material
interface. This group has made three presentations at recent high-profile meetings, and with future
publications of their recent work and unique materials growth capabilities could become among the most
highly recognized research groups in this field.
34

Among the applications addressed by the quantum science program is absolutely secure
communications, a major challenge for a highly mobile and ever-changing battle scene where the
opponent very likely will be able to receive one’s communications and thus unbreakable encryption is
essential. The ARL team has a significant effort on different approaches for a “quantum repeater” to
extend the range over which secure communications can be assured. The quantum sciences team is also
working on the full range of required components—quantum memories and single- and entangled-photon
sources and detectors, with a balanced approach looking at both ensemble and single-atom phenomena,
and within each of these, well-studied neutral atom and trapped ions to enable investigation of new,
Army-specific applications; and in parallel, the investigation of new materials that could provide far more
easily field deployable communications and sensor systems that are well beyond basic scientific
knowledge and laboratory demonstrations to enable their incorporation into Army applications. A second
important area is ultra-sensitive gravitational and magnetic gradient sensors. There is a good balance of
theory and experimental work with investigation into quantum oscillators with entangled sensors versus
independent atom oscillators that could significantly change the scaling of sensors and improve sensitivity
for quantum metrology.
There are three areas in which quantum sciences and entanglement can offer tactical advantages for
the Army—transfer of quantum information over long distances in a quantum network utilizing both fiber
and free space communications; precise timing, position, and navigation systems; and ultra-sensitive
gravitational and magnetic field gradient sensors. These are areas in which a small team at ARL can make
significant and scientifically leading contributions specific to the Army needs. This would avoid relying
on the much greater push of major university and commercial research organizations in quantum
computing.
following.
Quantum Networks: Many-Body Physics, Emergent Phenomena, and Applications of Distributed

Entanglement
This presentation described a theoretical project on quantum oscillators and basic condensed matter
physics and noise limits in quantum systems and particularly their application to entangled quantum
sensors. Quantum systems based upon neutral atoms, trapped ions, and tunneling are coupled over
distances of a micron or less. The theoretical modeling of entangled quantum oscillators over greater
distances will be required to achieve useful sensors based upon entangled states.
Divacancy Modeling in Silicon Carbide
This is an exciting new system that could be far more scalable and field deployable than other solid-
state quantum systems, but is highly dependent upon the energy difference of ground and excited states
and their immunity from outside perturbations and access for writing and reading entangled states. It is
critical to understand these basic properties of the system before investing too much experimentally.
Entanglement Swapping of Two Arbitrarily Degraded Entangled States
The development of robust quantum networks is challenging because nearly perfect entanglement is
required for many quantum networking functionalities. To address this challenge, the investigators’ team
has developed models and theories for the analysis of entanglement decoherence—both in transmission
35

and networking operations—and established tools to implement engineering trade-offs. The result of
these analyses provides a comprehensive method for analyzing entanglement swapping in quantum
networks.
OVERALL QUALITY OF THE WORK
In the energy-efficient electronics and photonics area, the impact of advances utilizing optical
equivalents, efficiencies realized through RF new waveform and encoding strategies, and efficiencies for
directed-energy applications are envisioned as being significant and important targets.
The quality of research in the MS&PP program is high and is showing continual improvement. It is
noted that ARL is among the country’s top-tier research organizations. Its research portfolio includes a
mixture of world-leading, established, innovative projects and recently initiated programs anticipating
scientific trends. Project investigators were generally clear presenting the linkage of their research to
Army needs along with connections to relevant research communities both within and outside the DOD
community.
anticipated that quantum sciences will provide game-changing capabilities for C4ISR for the Army of the
Overall, the researchers and the management are of high caliber and deserve kudos. Researchers
appeared passionate about their work. ARL’s work in preparing for the review was superb. The ARL
read-ahead materials, presentation viewgraphs, poster materials, and laboratory tours facilitated the
review process.
Most of the projects presented are excellent and have a pervasive impact. The scientific soundness
and the use of fundamental sciences are outstanding. The project portfolio fits well with both global
thrusts and the national agenda.
Presentations were of high quality, and most presentations addressed motivations, context and
specific performance targets. However, many posters and some oral presentations addressed only
scientific objectives, and objectives were not always measurable—that is, “to understand” a phenomena
or problem, as opposed to clear quantitative projection of how project outcome would contribute to goals.
In prior years, this issue has been raised (for example, in context of Heilmeier catechism), resulting in
presentation formats that explicitly capture these features in subsequent years, but this is not uniformly
retained year to year.
It would be useful if the description and presentation of ARL programs captured up front, clearly and
explicitly, exactly how the Army need differs from the trajectory of commercial off-the-shelf (COTS)
private sector technology, and how the ARL program is structured to address this difference. Future
descriptions and presentations would include intended pipeline, specify exactly where it departs from, and
leverage COTS to achieve targets—including handoff to 6.3, customer or vendor programs. It is often
hard to put things into context without this.
It is refreshing to see outstanding early-career professionals coming into the workforce and giving
excellent presentations. ARL does not seem to be suffering from any aging workforce challenges.
CONCLUSIONS AND RECOMMENDATION
The Open Campus Initiative is having significant impact on the culture of ARL. While initially
conceived as a way to bring outside participation into ARL resources, equally, if not more, important is
the complementary aspect of promoting contact between ARL researcher and the external scientific
community, which is providing ARL researchers perspective on their work. In discussions with the teams,
it was clear that many of these external collaborations were substantive and project-based, and valuable.
36

Overall, open campus is having positive impact on morale and is helpful in recruiting, both from the point
of view of outreach and visibility and in the working culture of the laboratory.
ARL has demonstrated ability to rapidly ramp new project areas (batteries and quantum sciences as
examples). ARL has also been doing outstanding hiring, and it is exciting to see ARL attracting its share
of the nation’s high-caliber talent. These are some of the facets of ARL’s notable advancement toward
becoming a premier scientific laboratory. In this process, it is appropriate and meritorious for ARL to
devote some additional resources to high-risk, potentially high-reward areas. It is important to note,
however, that high-risk work in the basic materials and device sciences will still benefit strongly from
parallel high-risk research on applications, and vice versa. This is distinct from a premature push of high-
quality discovery into application; instead, the application work is of sufficiently high risk to be rightfully
characterized as basic research as well.
There are previous national models of how successful in-house basic science, materials, and device
work can synergistically couple with in-house systems and applications research—the two linked to each
other in a tight feedback loop to drive rapid advancement, solving problems that the rest of the world has
not yet become aware of. As a relevant example, the global ramp of funded activity in quantum sciences
is driven in part by the prospective transition from science to application development.
Internal Army awards seem to be well targeted, but management needs to also focus attention to
external award opportunities, including aspiring for membership in the National Academy of Engineering
(NAE). More broadly, there is still strong opportunity to increase outreach and external visibility—ARL
is not as well known as it could be, which impacts perceptions and recruitment. This could include a more
compelling and useful web presence for the technical community.
anticipated that quantum sciences will provide game-changing capabilities for C4ISR for the Army of the
Recommendation: As ARL sets up its footprint in quantum technologies, it should emphasize

some concurrent fundamental application research to provide performance targets and test-bed
evaluation to steer materials and device development for sensors, communications, and
networking. Without this, there is risk of falling behind the leaders of the field.
37

Sciences for Lethality and Protection
The Panel on Ballistics Science and Engineering at the Army Research Laboratory (ARL) conducted
its review of ARL’s programs on battlefield injury mechanisms, directed energy, and penetration, armor,
and adaptive protection on June 5-7, 2017 at Aberdeen Proving Ground, Maryland. This chapter provides
an evaluation of that work.
ARL’s research in the area of sciences for lethality and protection during 2017 ranged from basic
research that improves our fundamental understanding of the scientific phenomena and technology
generation that supports battlefield injury mechanisms in human response to threats and human protective
equipment, directed energy programs, and programs that address weapon-target interactions and armor
and adaptive protection developments to benefit the warfighter. ARL’s breadth of Sciences for Lethality
and Protection Campaign work is performed within the Weapons and Materials Research Directorate
(WMRD), the Survivability and Lethality Analysis Directorate (SLAD), and the Sensors and Electron
Devices Directorate (SEDD). These directorates working collaboratively execute their mission of leading
the Army’s research and technology program and analysis efforts to enhance the protection and lethality
of the individual warfighter and advanced weapon systems.
BATTLEFIELD INJURY MECHANISMS
The study of battlefield injury mechanisms is a relatively new area of research at ARL, and ARL has
shown greatly improved coordination and focus over the past two years. Excellent progress on many
topics by both researchers and management was observed during the review. There has been considerable
improvement in prioritization of projects, at least in the near term; while some of the specific goals and
timelines of the remaining elements of the program need to be better developed and articulated. The ARL
focus on the definition of biological injury in materials- and engineering-relevant terms is a necessary
step in moving this critical area forward and may be unique in the field. Other groups studying this area
have concentrated either on the biology or the materials aspects but not on both. The focus at ARL on
identifying the critical size scale of injury is correct, and the emphasis on the translation of animal data to
humans is necessary and positive.
ARL is defining battlefield injury mechanisms in materials science and engineering terms,
differentiating itself from the medical community, and thus providing unique input to injury
modeling/simulation and armor requirements for injury prevention and assessment. Contributions from
researchers and management have significantly increased the focus and coordination of this highly
multidisciplinary team. ARL is establishing itself as a leader in this complex area, developing effective
collaboration with the relevant medical, materials and engineering communities both within and outside
the Department of Defense (DOD).
38

The focus of the battlefield injury mechanisms research group is to develop an understanding of the
translation of traumatic injury from ballistics or blast to the definition of injury mechanisms to allow the
design of improved personal armor. Extremity, head, sensory organs, and torso are all of interest, but
initial emphasis is on the combat helmet. The approach is to develop a human trauma model from porcine
data that ultimately results in an improved helmet design. Major differences between human and porcine
skull required transfer function definition of tissue and cell mechanical properties, and threshold
conditions for injury. It is assumed that the human and porcine heads will deform similarly, transfer data
to predictive model of ballistic injury, and confirm results on a second animal model (which was not
defined). The objective of the programs is a 20 percent reduction in weight of soldier armor—an
ambitious goal. Projects focus on development of a predictive ballistic injury model; identification of
critical injury size scale using pig data; measurement of mechanical properties of tissues, cells, and
biological molecules as required input to the injury model; and linking the animal data to humans to
predict human response. Overall, the area focus is well described, and integration of the various and
diverse technologies to achieve the stated goals is well formulated. However, timelines and resource
levels needed for success in defining brain injuries (or other ballistic induced trauma to the human body)
were not defined.
ARL management is complimented for creating a comprehensive program, with talented, energetic
scientists that cover the skills need to define and make progress in this area. This is an excellent start, and
ARL is encouraged to continue to grow this area.
Modeling
The modeling of force response related to ballistics for differing scales of injury is an appropriate use
of staff and resources to solve ARL-related issues for soldier injury. Most of the studies are currently
creating threshold baseline data to input into models related to molecular, cell, and tissue characterization.
ARL researchers are developing custom protocols and techniques to gather proper and relevant data into
the models for simulation of deformation of skulls, brain, and limbs. These tools could continue to be
supported and developed to help direct efforts toward soldier-related injuries. There seems to be a very
good working relationship with collaborators, who are able to provide access to equipment, expertise, and
data for input into the models. Overall, the modeling projects within the battlefield injury mechanisms
initiatives fit well with ARL priorities. The effort by the research staff and collaborators to produce the
proper computational tools shows potential for enabling a better theoretical mechanistic understanding of
damage caused by ballistic and blast forces on soldier injuries.
The project on modeling the porcine response to mechanical loading has developed a comprehensive
finite element model (FEM) based on material subroutines included in the commercial software,
LSDYNA, to simulate the response of impact on a helmet placed on a porcine head. The software is
certainly suitable for analyzing the problem if realistic material models and values of parameters in the
model are available. The project has modeled the helmet composite, the skin, the skull, and the brain
tissue with different material models. This effort has made significant progress in putting together various
submodels, studying an impact problem, and getting preliminary results.
The project on modeling surface waves in ceramic armor coupled the photon Doppler velocimetry
(PDV) diagnostic to the problem of material strength and fracture under dynamic impact relevant to
scenarios of bullet or fragment impact. The researchers on this project are commended for seeking a
collaborative effort that includes both experimentation and simulation. The development of damage
models—specifically, material fracture—is highly complex, but with significant impact toward the
design, development, and specifying requirements for personal protection equipment.
The development of a computational framework to investigate the effects of mechanical loading on
voltage-gated K+ and Na+ using molecular dynamics simulations for studying electrical transport between
neurons is relatively a small effort. While K+ and Na+ are not the ions of greatest interest in characterizing
neural damage, they are sufficiently characterized in experiments necessary for model inputs. This work
39

is largely proof-of-concept for a computational framework to conduct simulations at this scale. The
researchers are commended for considering damage markers across length scales in the larger context of
brain injury. However, this project was largely disconnected from the other projects due to the lack of
possible experimental comparisons and relatively small project funding.
Understanding behind helmet blunt trauma injury using the mini-pig model to study and quantify
trauma to human brain is a potentially powerful analogue. Custom apparatus (sensors implanted) and
techniques to make in situ measurements have been developed to gather data. A baseline model has been
created that can be used to evaluate human data. Clearly this fits into ARL core mission priorities to
provide the resources and staff to improve protective equipment standards for soldiers and discover
baseline data of actual forces that cause injury to the brain. This investigation has been successful in
showing that protocols can be developed with custom-designed equipment to measure comparable
locations of the force of a single impact on a helmet. The continued support of these types of projects that
develop tools can lead to applicable deliverables that could improve protection of soldiers and provide
leadership in adaptive characterization methods.
The anatomical variability and posturing modeling initiative makes an important contribution to
ARL’s modeling effort by providing a general method for modifying existing finite element models of the
human body to incorporate variations in gender, size, and posture to be expected in any real-world
scenario. This method is a cost-effective way to provide estimates on the variability of model outputs due
to these input distributions without laboriously remeshing the base model for each change. The project is
well conceived and competently executed. The work uses appropriate modeling tools and the computing
resources are adequate for the task. Results to date on both simple scaling of size and on more complex
positional changes are consistent with experience and warrant continued development and validation.
The investigation of helmet response to ballistic loading was undertaken to develop models and
methods to measure impacts on materials used in helmet protection. Clearly this fits into ARL core
mission priorities to provide the resources and staff to improve protective equipment standards for
soldiers. This project evaluates current ultrahigh molecular weight polyethylene (UHMWPE) and Kevlar
materials to establish baseline data to resist deformation caused by actual forces from impact and
penetration. A model was developed that correlates very well with experimental data for the blunt force
absorbed into the polymer laminate composites (delamination or ricochet depending on the angle of
impact). The model was predictive for flat panel versus curved structures to assess UHMWPE and
Kevlar. It is advisable to continue support for these types of projects that develop tools that provide
leadership in adaptive characterization methods and that could improve protection of soldiers.
The project on mechanics of soft armor is aimed at modeling the ballistic response of soft armor
(Kevlar) utilizing a finite element approach. The advantages of knit structures are the increased stretch of
the resulting fabrics, less mobility restrictions, and increased comfort when compared to woven
structures. Knit soft armor may offer significant improvements in extremity protection without sacrificing
soldier mobility. The finite element model developed has proven effective in predicting how changes in
fiber chemistry or knit construction impact the ballistic protection of the resulting soft armor.
Collaboration with the U.S. Army Soldier Research, Development and Engineering Center (Natick)
shows effective interactions with relevant Army laboratories.
Experimental
The experimental program investigating neuron health after being subjected to various strain and
strain rate environments as inferred from visible doping techniques showed an interruption of protein
transport from individual neurons following mechanical deformation. This project was part of a larger
effort seeking to understand damage markers at a cellular level for traumatic brain injury (TBI). The
problem is an ambitious, but potentially high-reward, investigation into a suspected critical aspect of
diagnosing mild TBI. Potential outcomes include a definition of cellular damage thresholds relevant to
Army-specific hazards. A clearly defined threshold for TBI that could be used to guide the design and
40

development of test protocols for personal protection equipment would be a transformative

accomplishment for future prevention of behind-helmet blunt trauma injuries. The state of the art in
diagnosing TBI is very rudimentary, and discovery of neuron response with mechanical insult is a
reasonable approach.
ARL has acquired state-of-the-art equipment for applying digital image correlation (DIC); this is
especially true for conducting stereo-vision DIC at high speed using the newest digital camera
technology. An example of DIC use is characterizing the constitutive behavior of the skull subjected to
ballistic loading conditions, which is a crucial step in the development of predictive models for blunt
trauma. The experimental program has developed a methodology to characterize the material strength of
skull bone matter by coupling DIC with the quasi-static compression experiments to obtain this type of
data. The multilayer aspects of skull structure suggest a modeling approach with discrete layers of
different strength. The diagnostic data collected is well suited to immediate use of continuum material
models (e.g., homogenized property definitions) or sophisticated mesoscale models of porosity and
multimaterials (e.g., spatially variant property definitions). The project is being conducted with clear
impact toward system-level simulations of behind-helmet blunt trauma injuries. The experiments have
successfully provided needed experimental data of material strength in surrogate (porcine) samples, and
the results have been extended toward human skull structure. There are many excellent examples of the
DIC being applied at ARL to a variety of experimental programs.
The research focus on the detection of impact-induced nano-structured changes in brain tissue is a
unique technique that describes the development of an ultra-structural analysis tool. The utility of this tool
within this study produces scan data using X-ray diffraction of a rat optical nerve and full brain slice
scans. These scans can be used to evaluate “ordered” biological specimens and, with contrast agents, “less
ordered” proteins. This study investigated the regulatory proteins that are ordered within single optical
nerve fibers. The goal was to measure and develop the ability to characterize nanometer space changes in
the layered nerve structures after impact.
There are different testing geometries between civilian data (car accident) and military data that do
not allow direct comparison of injuries to the lower leg. Thus, the Army has taken a leadership role in
investigating military-related injuries. The investigation of lower leg blast injuries was undertaken
through the design and development of methods to measure the impulse on lower leg simulants to provide
baseline data that could be used in the design of boot protection. Clearly, this fits into ARL core mission
priorities to provide the resources and staff to improve protective equipment standards for soldiers.
Multiple impacts (4) from a floor panel were studied on a control lower leg (no boot versus a boot) with
regard to lower leg injury. This investigation has been successful in showing that a properly designed
boot can protect against lower leg injury of a soldier in a neutral seated posture. The modeling program
related to these tests focused on ankle and tarsal fractures in foot location under high pressure. It is
advisable to continue support for these types of projects that develop tools that can lead to improved
protection of soldiers and provide leadership in adaptive characterization methods.
Quantification of the ballistic response of ceramics is an important area of study experimentally and
computationally, and one in which ARL is among the leaders in both experiments and simulations.
Taking a two-prong approach, better experimental monitoring of the performance of protective systems,
both brittle and ductile, coupled with the state-of-the-art computations, has increased Army tools for
prediction of armor behavior, with an emphasis on multihit phenomena. This is an area that is deserving
of strong ARL support, both with infrastructure and personnel, if both predictive design and performance
capability for ceramic armor is to be achieved.
Blast injury to the dismounted soldier, via shock coupling to the body of the warfighter, remains a
persistent and complex problem. Measuring the buried charge effects (primary) to test protective armor in
addition to aerosols and debris (secondary) is well justified. This fits into ARL’s core mission priorities to
provide the resources and staff to improve protective equipment standards for soldiers. This project
further evaluates high velocity soil propelled by buried explosives on the degradation of current uniform
fabrics. It reflects a blast injury problem identified by military surgeons in which ARL is uniquely
qualified to investigate. This study evaluated computational models for blast delivery velocities using a
41

shock tube and fabric model simulations that generated data from experiments to input into soil and fabric
models. This allowed for development of standard blast methodology that can be used to develop new
fabrics.
ARL is to be complimented on having created a well-focused, unique, multidisciplinary research team

focused on a critical, highly complex, multifaceted research area. The overall coordination and focus of
work has greatly improved over the past two years. ARL needs to continue to focus on coordination
across this area. The emphasis on identification of critical size scale of injury is correct, and further
emphasis in this aspect represents an opportunity for further study. ARL focus on the definition of
biological injury in materials and engineering terms may be unique in the field and is needed to move
injury protection forward.
ARL’s research group is maturing but has not yet developed as a cohesive group. Therefore, a
continued focus on team building and diverse competencies is needed. Integration of computation and
experiment appears to be well done, but the level of modeling and its integration in the research does not
seem as sophisticated as that in other areas within ARL’s science and technology (S&T) portfolio.
Increased emphasis on modeling in the battlefield injury mechanisms projects is strongly suggested.
The battlefield injury mechanisms portfolio seems well connected to outside groups, although the
nature of collaborations was unclear. ARL needs to pursue activities aimed at impacting and organizing
the larger external community in this area, taking a leadership position in sectors of the injury
mechanisms S&T area within the DOD and the larger academic community. As one example, ARL’s
focus on quantifying and translating animal data to humans appears strongly warranted, necessary, and
positive. ARL is probably a leader in this aspect of the field and could represent one area for ARL to lead
the external research community. Two types of pigs were discussed, and the response of the two types is
not the same. Therefore, there is the challenge of pig-to-pig transfer function as well as pig-to-human
transfer function. This approach of developing transfer functions is very important for making substantial
progress on inferred injury mechanisms to humans based on animal studies.
A challenge to the battlefield injury mechanisms team lies in finding a common language that
transcends the disciplinary range of the team. The opportunity is to provide unique solutions to the
prevention of injuries caused by nonpenetrating ballistic impact such as traumatic brain injuries and
compressive force injuries to the extremities. Effective definition of priorities, timelines, and strategic
goals in this complex area are continuing challenges.
Finally, ARL’s battlefield injury mechanisms programs have reached the level of maturity where
timelines, well-defined goals, and a well-defined overall program roadmap are critical to guide future
work.
Modeling
Modeling the porcine response to mechanical loading is very challenging, and its satisfactory solution
requires expertise in the areas of constitutive modeling, estimating values of material parameters from test
data (assuming that it is available under test conditions likely to prevail in the impact event), numerical
analysis, and interpreting results of numerical simulations. The development of the software, LSDYNA,
is based on several assumptions. The user needs to be aware of these assumptions and various subtleties
in using the software. Once the use of the software has been mastered and reasonable values of material
parameters and so on have been determined from test data, then it will be a powerful tool not only for
analyzing and ascertaining the damage induced by impact loads but also for designing new armor.
In the project on modeling surface waves in ceramic armor due to the stochastic nature of brittle
fracture, experimental design of model validation is a challenge. In this case, the burden is on the
42

experimentalist to quantify experimental uncertainties. The large-model development effort presented in

the project on ballistics response of ceramics could be reevaluated to better leverage the modeling
capabilities of the ALEGRA computational framework with the experimental conditions uniquely of
interest to the Army.
The development of a computational framework to investigate the effects of mechanical loading on
voltage-gated K+ and Na+ using molecular dynamics simulations for studying electrical transport between
neurons represents an intriguing scientific challenge. The largest challenge to this project is related to the
lack of experimental data for the Ca+ ion, the ion of major interest, and conducting simulations at a scale
sufficient to draw conclusions about the operability of systems of neurons. Connecting the outcomes from
this project (at molecular scale) to the project studying porcine brain response (at the single-neuron scale)
is currently not possible. The research framework may benefit from a reevaluation of the best use of ARL
resources and team skill sets.
Understanding behind-helmet blunt trauma injury using the mini-pig model to study and quantify
trauma to the human brain is a potentially powerful analogue. The major challenge is the comparison of
the animal model to what may occur in a human. There is a need to discover what forces are causing
micro-tissue and vascular damage even though the helmet can successfully stop the projectile. Also, there
is a need to plan and develop infrastructure for survival surgeries of animal models to discover long-term
outcomes. There are opportunities to try to change the protocols to measure animal survival health and
possibly lead to an investigation of what areas of the brain are being affected by the injury (motor
functions). Data measured can be used to guide new standards in characterization of blunt trauma injury
and development of models to improve animal-to-human correlations.
In the anatomical variability and posturing modeling thrust, although the level of effort is modest,
considerable thought could be given to a thorough validation protocol given the expected widespread
adoption of this adaptation to human models for many purposes.
The investigation of helmet response to ballistic loading was undertaken to develop models and
methods to measure impacts on materials used in helmet protection. The testing of materials under
varying environmental conditions (temperature, etc.) could also be explored to discover changes in
material properties as a function of the environment. There are opportunities to develop a high throughput
assessment of newer polymers and composites for helmet protection, and to compare these to existing
materials being used in current helmets. Baseline data measured can be used to guide new standards in
characterization of blunt trauma helmet protection models.
The project on mechanics of soft armor is aimed at modeling the ballistic response of soft armor
(Kevlar) utilizing a finite element approach. The project may benefit by teaming with university research
groups with extensive fabric mechanics expertise—for example, North Carolina State University or
Clemson University.
It needs to be noted that modeling the response of ceramic armor is very important, and extremely
challenging, given that ceramic armor has been studied since the late 1960s. Although progress has been
made in fracture initiation and propagation, a robust and highly accurate computational ceramic
constitutive model for multiple impacts remains elusive. The same can be said for some of the substrate
materials (e.g., UHMWPE, Kevlar, etc.). Some of the new diagnostics described in the “Penetration,
Armor, and Active Protection” section have promise in elucidating fundamental mechanics of failure and
propagation.
Experimental
Significant challenges exist in terms of relating neuron response to the manifestation of post-
traumatic stress disorder, mild TBI, and severe TBI that is challenged further by limited staff availability
and skills. Due to the problem complexity, progress may be likely only from a focused research thrust that
is supported through a team of researchers with a multifunctional skill set consisting of microbiologists,
biomedical engineers, shock physicists, and mechanical engineers. While the individual researchers could
43

communicate the interconnectivity of different research projects, as a whole, the effort appeared too small
(both in terms of staffing and funding) to make meaningful contributions to this field within a short time
frame. The team stated that funding for the current multiyear program was ending this fiscal year. There
are significant opportunities to capitalize upon the recent emphasis toward mitigating additional TBIs, the
unique Army hazards, and the immature state of the art for ARL to dominate the research of TBIs from a
materials and engineering perspective. ARL is at the cusp of establishing a transformative research
program coupling engineers, microbiologists, and physicists advancing experimental methods and
computational models for the future prevention of behind-helmet blunt trauma injuries. ARL’s program
would benefit from a larger strategic effort with transparent and clearly articulated goals that closely
couple the Army medical researchers, the ARL research team, and other external researchers (e.g., sports
injury researchers), perhaps within a multiagency research consortium. Some brief mentions were
provided of external collaborations, but presented information remained largely focused on the local ARL
researchers, resulting in an assessment that potential programmatic reward will be limited in scope.
Characterizing the constitutive behavior of the skull subjected to ballistic loading conditions is a
crucial step in the development of predictive models for blunt trauma. Challenges exist in relating quasi-
static material properties to the simulation of dynamic impacts along with the development of material
strength models suitable for representing the large human population from singular experiments. The
applicability of the material strength models being developed under low strain-rate conditions could be
considered relative to the strains expected under conditions of behind helmet blunt trauma. A useful
undertaking would be for ARL representatives to participate within the external DIC community. Also,
ARL needs to increase its focus on uncertainty quantification. Efforts toward understanding the interplay
between experimental factors and the measurement uncertainty of quantities were largely unreported.
Characterizing skull material strength has the ability to impact the warfighter directly through improved
personal protection equipment and also the larger knowledge base of human physiology. ARL’s unique
position to contribute to this understanding from a mechanics of materials perspective is unrivaled by
other agencies. It would be beneficial for ARL to lead the experimental research in this effort. Because of
the mesoscale variations of skull materials, the ability to advance model development from homogenized
properties toward spatially resolved predictions could be exploited by growth in the area of mesoscale
computational model development.
The research focus on the detection of impact-induced nano-structured changes in brain tissue is a
unique technique that describes the development of an ultra-structural analysis tool. A significant
challenge to this work is the measurement of the shear forces on the structures within the brain and the
resulting mechanical response to tissue degeneration. The next steps could include the investigation and
characterization of the compromised regulatory structure force response to translate from the single fiber
structure (nerve) to bundle (nerves) to brain tissue model. The project may also find it fruitful to
investigate control/direct loading versus impact loading on nanometer space changes between layers.
There is extensive research being conducted related to football and other sports injuries on TBI and ARL
needs to be knowledgeable of and strongly coordinating with that community.
The investigation of lower leg blast injuries was undertaken through the design and development of
methods to measure the impacts on lower legs to provide baseline data that could be used in the design of
boot protection. Trying to institute new boot standard specifications may not be popular with troops.
There are opportunities to try to scale models of boots desired by troops for comfort and protection. Data
measured can be used to guide new standards in specification of military boot to provide protection from
injury.
Blast injury to the dismounted soldier, via shock coupling to the body of the warfighter, remains a
persistent and complex problem. The challenge is to measure the injuries sustained by a soldier due to
deteriorated fabric exposed to debris. There are opportunities to study operationally relevant experiments
to measure shock and debris kinetics on fabrics. This also allows the investigation of the critical role in
the failure of fabrics due to wear patterns to material type and distance from the blast.
44

DIRECTED ENERGY
The ARL directed energy (DE) program focuses on radio frequency (RF)-DE and laser-DE. ARL
leadership and research teams successfully implemented some of the recommendations of the previous
Army Research Laboratory Technical Assessment Board (ARLTAB) report. Specifically, the collection
of presented projects demonstrated a coordinated strategy across the enterprise for the laser-related DE
work, with indications of much greater collaboration with the Navy and Air Force. Internal to ARL,
principal investigators (PIs) from the DE sessions demonstrated greater awareness of work done with
threat warning and countermeasures. The quality of the programs will continue to benefit from even
deeper and more frequent collaborations both internal and external to ARL to foster rapid innovation with
operational and contextual relevance.
The DE work presented aligns with the overarching goal of developing new knowledge and
technologies associated with lowering size, weight, power and cost (SWAPC) of army-vehicle mounted
offensive and defensive DE systems. Low size, weight, and power (SWAP) improvements for the
protection and hardening of platform apertures were also investigated.
Although ARL is not working to develop fully functional DE systems, the work presented aims to
make unique impacts in the areas of compact power, propagation, and target response. Other novel or
high-risk areas of work include beam control, thermal management, and material effects for Army-centric
operational scenarios and demonstrations. Use of high peak power ultra-short laser pulses to “burn” their
way through the air and obscurants to create quasi-steady-state waveguides called filaments is a complex,
high-risk area of basic research. The mechanism is based on super-heated air (plasma) depositing energy
in the air causing heat-induced changes in the index of refraction on the order of millisecond time scales.
Secondary pulses or quasi-continuous wave (CW) DE propagating along the same path “sees” the thermal
disruption path as a lower loss waveguide.
In contrast, the overall strategy for the RF-DE work was not as evident based on the topics presented.
Specifically, The RF work appeared to remain fragmented and unrelated to the overall mission. For
example, the operational context of the cognitive radar and the counter-unmanned aircraft system (UAS)
work was not apparent. Progress made over the course of the last two years was not evident in two
projects—cognitive radar and counter UAS.
Overall, the DE research program demonstrated positive trending in adding value to body of research
and development (R&D) knowledge with relevant operational focus and alignment to the ARL enterprise
strategy.
Some ARL researchers are pursuing complex, high-risk areas of basic research, and this is
encouraged to continue. This work includes novel waveguide concepts investigating a high-risk, low-
technology readiness level (TRL) technique to improve laser propagation through atmospheric mediums
using high peak power ultra-short laser pulses to “burn” their way through the air and obscurants to create
quasi-steady-state waveguides called filaments. Further, ARL presented two additional projects that
encountered materials properties challenges that the principal investigators (PIs) were aware of, and they
are encouraged to be persistent in pursuing solutions. Those projects focused on low-loss fiber and ultra-
low quantum resonantly pumped illumination.
Particularly outstanding laser DE work is being performed. The work reported on exploiting Raman
lasers to greatly improve fiber power output is exceptional and is an archetype for research at ARL that
compliments other DOD laboratories while not duplicating academic or industrial research. The key
evaluation metrics were exceeded in the presentation of this work and include excellent contextual
reference with alignment to the overall strategy; definition of quantitative objectives based on models
(e.g., using the chart that identified Raman gain predictions); and establishing aggressive benchmarks for
success and demonstrating consistent progress. Another example of exceptional work was multiple
45

chromophore mixtures for sensor protection focused on technology to protect and harden the sensors on
Army assets against DE threats. The method uses nonlinear optical materials and coatings that are
frequency agile in the visible spectrum to passively protect Army optical sensors from DE-laser threats
such as that from straight damage, jamming, dazzling, and so on.
ARL has made substantive progress over last 2 years, addressing many of the shortfalls identified in
previous ARLTAB reports, including increased collaboration with a diversity of research partners to
validate and compare results or address experimental and modeling gaps; growth in the knowledge depth
of its personnel; prioritized research based on operational relevance in streamlined focus areas;
establishing a strategic vision by looking at operationally relevant drivers and trade-offs to determine
where discovery is really needed; system-level or event-driven discovery rather than disparate seemingly
random and siloed projects. While most of these improvements directly apply to the laser-DE program,
the overall quality of work is good and on an upward trend with examples of exceptional work.
The overall scientific quality of the research is comparable to that at leading national and international
institutions, with several projects demonstrating impressive quality and uniqueness with realistic potential
for high payoffs, such as diode clad pumped broadband Raman fiber lasers working to achieve potential
power scaling in the ballpark of 80-100 KW from a single fiber aperture, and nonlinear optical materials
and coatings that are frequency agile in the visible spectrum to passively protect army optical sensors
from DE laser threats such as that from straight damage, jamming, dazzling, and so on.
ARL demonstrated a willingness to push and even lead the technological edge by pursuing a focused
selection of exciting, high-risk work that the ARLTAB encourages ARL to continue. Select laser-DE
work demonstrated exceptionally disciplined research approaches, while some projects could benefit from
a more disciplined approach to model validation. More fundamentally, RF work can benefit from vision
and focus demonstrated with laser work in order to show continuity and measureable progress.
An exceptional accomplishment is the work on exploiting Raman laser to greatly improve fiber power
output and is an archetype for research at ARL that compliments other DOD laboratories while not
duplicating academic or industrial research. Additional exceptional work is the nonlinear optical materials
and coatings that are frequency agile in the visible spectrum to passively protect Army optical sensors
from DE-laser threats such as that from straight damage, jamming, dazzling, and so on.
In order to provide a more comprehensive assessment of research activities, improvements in the

presentation logistics and structure in future ARLTAB panel meetings is needed. The ARLTAB had a
difficult time in assessing two of the major components of the evaluation, specifically “quality of the
research” and the “qualifications of the research team.” Therefore, it was difficult to make those
assessments when some principal investigators were not present. Substitute presenters did well but were
unable to answer detailed questions—for example, on cognitive radar; lasers through the atmosphere; and
efficient Er-based mid-infrared (mid-IR) laser. Essential classified material or underlying classified
concept of operations needs to be presented in classified venues—for example, on counter-UAS,
ultrashort radar, and cognitive radar. Presenters need to clearly articulate the contextual reference by
including the operational context, research goals, and progress toward more model-based benchmarks. By
following the model of those presentations identified as exceptional, future presentations may include
quantitative objectives or analytically derived requirements.
The positive trend toward operational focus could be applied to the RF-related work. The RF work
appeared to remain fragmented and unrelated to the overall mission. For example, the operational context
of the cognitive radar and the counter-UAS work was not apparent.
There are indications of much greater collaboration with Navy and Air Force. In addition, internal to
ARL principal investigators from the DE sessions were aware of work done with threat warning and
countermeasures. However, greater collaboration could uncover additional innovation and is strongly
encouraged.
46

Several of the projects lacked quantitative objectives or analytically derived requirements—for

example, counter-UAS lacked tactically significant range targets. The projects could benefit from a more
disciplined approach to model validation.
The work on utilizing the thermal signature of an ultrashort pulsed laser filament for guiding fiber
lasers investigates a high-risk, low-TRL technique to improve laser propagation through atmospheric
mediums using high peak power ultra-short laser pulses to “burn” their way through the air and
obscurants to create quasi-steady-state waveguides called filaments is a complex, high-risk area of basic
research. The work is encouraged to continue maturing novel waveguide concepts.
Some key opportunities to enhance the DE research program remain. There is a greater attempt to tie
the projects to a coherent set of mission areas; however, the mission-driven strategy alone is not
sufficient. There needs to be a tighter coupling of the research topics to the overarching problems that
ARL has chosen to investigate. When asked how projects were prioritized, the answer was incoherent,
with support for demonstrations being the key drivers for the research. This was illustrated in the opening
presentation describing swarm, counter-swarm and counter-UAS. The demonstrations were nice but were
lean on fundamental research objectives. Model validation continues to be an ad hoc process in many of
the projects. Without contextual reference, the study of laser eye safety in an aerosol environment could
await system development.
PENETRATION, ARMOR, AND ADAPTIVE PROTECTION
ARL continues to demonstrate a strong record of achievement in the fundamental and applied
sciences and the engineering of penetration, armor, and adaptive protection. The ongoing work described
in the 2017 review continues to highlight how ARL is building on its history of excellence to provide the
knowledge basis for future Army needs in the area of warfighter protection. This is an absolutely critical
and core competency that underlies Army capabilities.
A few years ago, there were severe restrictions on attending and participating in technical conferences
and symposia. It was good to see that this disturbing trend has been reversed, and the staff is now
obtaining approval to present elements of its technical work at venues such as the Society for
Experimental Mechanics, the American Physical Society’s Shock Physics Conference, the International
Symposium on Ballistics, and the American Ceramic Society’s annual exposition and conference. These
and related conferences are great settings for professional development, for the staff to present its work
and receive feedback from technical peers, and engage in technical discussions. Such technical dialogues
often lead to further insight, and spark new ideas and avenues of investigation. Management needs to
continue to encourage participation in technical societies and associated meetings, as peer review and the
ability to always know where the state of the art is moving is critical to this area, especially as so much of
the leading-edge research is classified or limited-release.
A major research initiative has been launched as the result of a war-gaming exercise that identified a
potential vulnerability. A well-developed, multifaceted research effort has been established and is being
executed. The advanced penetrator work is a potential game-changer and is viewed as an exceptional
accomplishment. Integral to this initiative is the use of classified computations to support analysis of
possible mitigation concepts. ARL researchers have taken high-performance computing (HPC) to a whole
new level—an impressive display of using computational sensitivity analysis of a very, very large
problem with multiple length scales. The sensitivity analyses have resulted in identification of unknowns
and issues, quantification of design variations, and establishing where basic information is required for
higher fidelity simulations. As examples, the computational analyses have identified where fundamental
information is lacking, such as thermal conductivity at elevated temperatures in a plasma. It has been
47

determined that computational results are dependent upon the calculated temperature distribution, and a
parametric study of various constitutive strength models showed that the prediction of temperature as a
function of time and location depends upon the assumed model.
Although the computational study has identified weaknesses, and fundamental research initiatives are
being formulated, decisions are required in the short term to address and mitigate the identified
vulnerabilities. This, then, is a significant challenge faced by ARL researchers. A balanced technical
effort of HPC combined with well-planned experiments with appropriate diagnostics could provide
insights and guidance for progress while a longer-term research effort addresses the fundamental
unknowns.
It is interesting to note that the goal of ARL is to conduct the research required to sustain the Army 30
(or 50) years in the future. Yet, the preceding is a major initiative that has a relatively short time frame.
Clearly, rapid response to uncovered threats and vulnerabilities needs to be (and is) a part of ARL’s
portfolio; thus, the mission statement could reflect the realities of the real world, while emphasizing the
need for long-term research.
Modeling
As part of their HPC initiative, ARL has begun to quantify uncertainties in computational results.
Uncertainty quantification (UQ) was not part of the ARL technical initiative one or two years ago, and
ARL was criticized by the ARLTAB for the lack of UQ. ARL has responded, and has demonstrated a
good start. As an example, researchers examined the uncertainty in computational results for two state-of-
the-art transient (dynamic) wave-propagation, solid mechanics computer programs. The researchers
determined the uncertainties in the results through parametric studies, which included changes in material
parameters and changes in the computational domain (for example, changing mesh size and parsing the
problem to different processors). While some results were not surprising (for example, the depth of
penetration depends upon the strength of the target material), the researchers found one example where
the results differed greatly as the computational domain was adjusted. The root cause was that the
algorithm to distribute the computation mesh onto multiple processors was not invariant to how the
domain was distributed to the various processors. This issue is being addressed, but it demonstrates the
importance of UQ.
Diagnostics and Collaboration
The advanced diagnostics work at ARL has great potential in helping to bring illumination and insight
to unresolved problems in dynamic response and failure of traditional and advanced materials. One
example is the quantification of material strength at high strain rates by constructing smaller and smaller
split-Hopkinson pressure bars (SHPBs). The smallest SHPB utilizes specimens 50 microns in diameter
and 25 microns long, and can achieve strain rates of 5 × 105 s-1. Challenges have included development of
instrumentation to interrogate the response of the specimen; achieving linearity of the input bar,
specimen, and output bar; and even preparing specimens. For example, on a somewhat larger system than
the micro SHPB, specimens were prepared by milling or centerless grinding. Since the samples were
small, the relative volume of work-hardened material was high, which significantly strengthened the
smaller samples. Samples are now prepared through electrical discharge machining, and the sudden
increase in strength at very high strain rates is no longer observed for some specific materials.
Another example of advanced diagnostic work at ARL is the development of time-resolved, in situ
imaging. ARL researchers are conducting experiments using phase-contrast X-ray imaging at the
Advanced Photon Source (Argonne National Laboratory [ANL]). This is a unique, national resource,
which the ARL is taking advantage of. The objective is to record in situ observations of microstructural
evolution, damage, and fracture in terminal ballistic events. Also, the high-voltage in situ diagnostic
48

radiography apparatus (HIDRA), which consists of 14 flash X-ray sources and includes two synchronized
ultra-high-speed cameras used in stereo imaging, has been developed and is being applied to impact of
ceramic targets. Integral to the effort is quantitative comparison with computational simulations, which
has shown some agreement, but has also highlighted important differences near the impact site, which
suggests deficiencies in the understanding of the failure processes. An advantage of HIDRA is that it is an
in-laboratory resource that is not limited to availability and scheduling of the Advanced Photon Source
(however, see the following concerning the availability of range time).
A collaboration with Ernst-Mach-Institut and Lawrence Livermore National Laboratory has the
objective of developing a multienergy flash X-ray computed tomography diagnostic. This diagnostic,
once completed, will image events in three dimensions and at three points in time. At this point, a
tomogram reconstruction has been accomplished using only four source/detector pairs. Already lessons
have been learned about shielding and X-ray film requirements for quantitative assessments.
Penetration Mechanics and Adaptive Protection
Adaptive protection is a subcategory of the Sciences for Lethality and Protection Campaign, with the
objective of developing mass-efficient, novel threat defeat mechanisms.
One project is notable because of its success while highlighting a concern. The study used a modeling
and simulation tool set that had been developed over the last few years to facilitate mine blast simulations.
The technical effort was an HPC study to investigate and optimize a cellular core structure that
demonstrated considerable enhanced performance over legacy structural systems to mine blast. At this
point, no experiments have been conducted, but an Army partner is planning a full-scale demonstration
experiment.
Another subcategory of the Sciences for Lethality and Protection Campaign is penetration mechanics.
A novel adaptation of fundamental penetration mechanics has the potential of being a “game changer” in
terminal ballistics and lethality of long-rod projectiles. Approximately one-quarter-scale testing has
demonstrated greatly increased penetration, and a few design iterations have been devised to control
unwanted and undesirable effects.
In 2014, ARL reorganized its research portfolio into four functional campaigns: Human Sciences,
Sciences for Lethality and Protection, Sciences for Maneuver, and Informational Sciences. ARL also
identified four crosscutting campaigns: Extramural Basic Research, Computational Sciences, Materials
Research, and Analysis and Assessment. Within a functional campaign, ARL has identified essential
research areas (ERAs). This reorganization has permitted ARL to develop an integrated, multifaceted
research program to address current and future Army requirements. There are a number of examples of
researchers spread among four or five projects, but there is commonality because the projects are related
or integrated into a bigger picture. This assists in communication and cross-fertilization. However, some
researchers in this area noted that they are spread among four or five projects and that this results in a
decrease in depth of the work. This “integrated” approach can be contrasted to researchers being spread
over a number of projects that have little-to-no commonality; consequently, the researchers are now
spread thin with the result of decreased depth in their research. ARL management is encouraged to weigh
the trade-offs that each approach offers.
A few items were identified that the ARL staff found to be especially irritating and are impediments
to productivity. In particular, the Army procurement system is not designed to handle efficiently the
requirements of procurement for a research environment, where items are purchased in very small
(sometimes single-unit) quantities. Several researchers identified purchasing as a significant impediment
to research. To purchase research equipment like a laser takes about one year due to the “red tape.” It
49

appears that the process for purchasing research equipment is the same as requisition for the Army in
general. There were instances cited where it took a year from submitting a purchase request to receiving
the item. Progress grinds to a halt if that item is an essential piece of equipment for an experimental
program (for example, a laser with specific requirements). Although it is recognized that this is an issue
that is beyond the control of ARL, ARL needs to aggressively pursue an approach for developing
streamlined procurement procedures applicable to a research environment.
Some operations (e.g., debugging diagnostics) could likely be performed safely with two people, as
would be required for indoor laboratories. The number of personnel required for each type of firing-site
operation could be reviewed by management to explore avenues to improve efficiency and productivity
without increasing safety concerns.
Some important research initiatives are being delayed because testing ranges are shared between
several projects. This is not a complaint that people cannot work together, but various projects have
different range and diagnostic requirements. Sharing range time often means tearing down the diagnostic
setup to permit the next project to set up its respective diagnostics and conduct experiments. Thus, setups
and teardowns disrupt continuity; progress is delayed by not having access to the range while another
group is conducting its set of experiments. This greatly affects productivity. ARL management is
encouraged to investigate what can be done to allocate dedicated ranges to some of its important efforts in
the penetration, armor, and adaptive protection program areas.
There is a start of work in the UQ area of research, but there is a long way to go. It was unclear,
however, what level of research is being conducted at ARL, or the number of people involved. ARL
needs to continue an emphasis in UQ. Further, ARL could purse the integration of UQ into its data-to-
decision workflow that includes modeling and simulation, experimentation, and design. To accelerate
integration and given the complexity of its objectives, ARL scientists and engineers could leverage
software and methodologies developed at other Department of Energy (DOE) and DOD laboratories.
Regarding the development of anti-tank guided missile (ATGM) soft-kill countermeasures, the
current staff has a physics background, which is insufficient for making rapid strides in engineering and
development of the project. Perhaps engineers with suitable backgrounds could be recruited to facilitate
rapid design and development.
ARL needs to resist going to full-scale demonstrations without any or minimal small-scale testing.
Such a rapid rush to conduct “admiral’s tests” limits development of core competency in terms of
fundamental knowledge. Further, integrated full-scale testing without the underlying physics
understanding does not develop predictive design capability or accurate performance modeling.
Additionally, the rush to full-scale testing can lead to very expensive surprises and unforeseen application
problems. The cellular core structure underbody blast application was noted as a concern in this area. The
structure was designed on computational science. However, the fabricated structure cores were never
tested for material response at small scale to verify if the assumed response was obtainable. No
fundamental study or validation experimental was performed before the structure was sent for field
testing.
Modeling
Thus, ARL has a good start in UQ, where none had existed a couple of years ago. The concept of UQ
needs to be embraced by all the researchers and integrated into their data-to-decision workflow, which
includes modeling and simulation, experimentation, and design. To accelerate integration and given the
complexity of the objectives, ARL scientists and engineers need to leverage methodologies and software
developed at the DOE weapons laboratories.
ARL embraced one of the suggestions of the ARLTAB by establishing an ERA to understand failure
of materials, explicitly failure of materials under high rates of loading and, often, high pressures. With
understanding, the objective then becomes one of designing materials to manipulate failure and to exploit
mechanisms. Predictive modeling of failure requires insight into initiation, nucleation, and propagation
50

mechanisms, which occur at various length and time scales that often differ by orders of magnitude. An
essential element to making substantial progress in the physics of failure is the developing of advanced
diagnostics that can interrogate material response in extreme dynamic environments.
Diagnostics and Collaboration
The current work using the very small SHPBs, which includes aluminum, Ti-6Al-4V, and tantalum,
could be expanded to include a face-centered cubic (FCC) metal (such as FCC copper or a FCC steel),
which others have shown to have strong strain-rate strengthening at rates above 104 s-1. This work needs
to continue, and results compared with plate impact and pressure/shear experiments. This will probably be
challenging in interpretation because of the high pressures in plate impact experiments and the use of
different diagnostic measuring devices.
Much remains to be done for the X-ray tomography diagnostic that is being developed, including
modification to include spectral analysis, which will allow identification of material type by using three
different energy broadband sources. The photon flux versus energy of the sources needs to be established
and the algorithms need to be developed to compute material specificity, with all being integrated into
post-processing software. The challenges are nontrivial, and it is highly likely that they will require a
sustained effort as unanticipated problems arise and are resolved. However, this diagnostic, once on-line,
will be a unique addition to ARL’s suite of experimental diagnostics to interrogate the in situ response of
materials in an extreme, dynamic environment.
Penetration Mechanics and Adaptive Protection
Adaptive protection is a subcategory of the Sciences for Lethality and Protection Campaign, with the
objective of developing mass-efficient, novel threat defeat mechanisms. The challenge is real-time threat
sensing and identification (while minimizing false positives), and defeat of the threat. Individual projects
have little cross-fertilization (the nature of these small- and medium-funding level projects are very
diverse, with the primary commonality being to neutralize a threat). Some of the projects that fall into this
category include the optical threat warner, ATGM soft-kill countermeasures, trajectory alteration, and
tailored explosive response. Some of the projects have well-integrated numerical simulations and
experiments, while others appear to be demonstrations with little theory or advancing fundamental
knowledge. These projects tend to be “stove-pipe,” in contrast to a well-integrated, synergistic research
plan like humans in extreme ballistic environments or manipulating physics of failure. While the technical
efforts fall under the purview of ARL, many small, remotely connected projects provide management
challenges for a well-integrated program.
One project used a modeling and simulation tool set that had been developed over the last few years
to facilitate mine blast simulations. The caution is to resist going to full-scale demonstration prior to some
confirmation experiments. This work could be linked to an experimental program that is done at smaller
scale to validate assumptions in the computational model and to ensure performance of the cellular core
structure. ARL could negotiate with the Army partner to move forward a little more cautiously.
Another subcategory of the Sciences for Lethality and Protection Campaign is penetration mechanics.
The ARL is encouraged to put an emphasis on this novel approach with continued small-scale testing to
explore robustness, and uncover and resolve potential issues. Numerical simulations currently are not able
to replicate accurately the experimental results, and a priority could be given to resolving this issue. Initial
full-scale testing could be implemented on select designs to ensure that issues with scaling up the concept
are understood and resolved.
51

In the battlefield injury mechanisms area, the integration of experiment and modeling is well done,
although the level of modeling sophistication does not seem to match that of other ARL research groups.
The group is at an early stage of defining key mechanical impact limits of injury, and this has been
correctly identified as a critical end point for the group. The ARL team is to be complimented on having
built effective bridges to outside research groups both within the Army and with universities. ARL needs
to explore collaboration with investigators studying football and other head injuries. By nature of their
approach, ARL is assuming a leadership position in this difficult and critical area of study. Focusing
efforts on brain injury and helmets is justified and allows research focus in a very broad area of study.
Overall, the group is talented and enthusiastic, and is maturing as a team that has not reached its full
potential. Continued focus on team building and definition of deliverables and criteria for acceptable
progress is still a work in progress. The scope of the work undertaken by this group is very broad, and it is
not apparent that sufficient resources, especially in key areas of biological and computational sciences,
are available. There is an apparent issue in communication between the various disciplines in this highly
interdisciplinary research group (generally true when bringing physical, biological, and engineering assets
together). This represents an opportunity for ARL to catalyze educational programs designed to facilitate
communication between diverse scientific disciplines.
In the DE area, the overall scientific quality of the research is comparable to that at leading national
and international institutions, with several projects demonstrating impressive quality and uniqueness with
realistic potential for high pay-offs. The DE program focuses on radio frequency (RF)-DE and laser-DE.
Internal to ARL, principal investigators from the directed energy sessions demonstrated greater awareness
of work done with threat warning and countermeasures. The quality of the programs will continue to
benefit from even deeper and more frequent collaborations both internal and external to ARL to foster
rapid innovation with operational and contextual relevance. Specifically, the collection of presented
projects demonstrated a coordinated strategy across the enterprise for the laser-related DE work, with
indications of much greater collaboration with the Navy and Air Force. In contrast, the overall strategy for
the RF-DE work was not as evident based on the topics presented. Specifically, the RF work appeared to
remain fragmented and unrelated to the overall mission. For example, the operational context of the
cognitive radar and the counter-UAS work was not apparent. Progress made over the course of the last
two years was not evident in two projects—cognitive radar and counter-UAS.
In the penetration, armor, and adaptive protection area, ARL continues to demonstrate a strong record
of achievement in fundamental and applied sciences as well as engineering. The ongoing work described
in the 2017 review continue to highlight how ARL is building on its history of excellence to provide the
knowledge basis for future Army needs in the area of warfighter protection. This is an absolutely critical
and core competency that underlies Army capabilities. A major research initiative has been launched as
the result of a war-gaming exercise that identified a potential vulnerability. A well-developed,
multifaceted research effort has been established and is being executed. Integral to this initiative is the use
of classified computations to support analysis of possible mitigation concepts. ARL researchers have
taken HPC to a whole new level—an impressive display of using computational sensitivity analysis of a
very, very large problem with multiple length scales. The sensitivity analyses have resulted in
identification of unknowns and issues, quantification of design variations, and establishing where basic
information is required for higher fidelity simulations.
CONCLUSIONS AND RECOMMENDATIONS
ARL’s research on sciences for lethality and protection ranges from basic research that improves its
basic understanding of scientific phenomena to the generation of technology that supports battlefield
injury mechanisms, human response to threats, and human protective equipment; directed energy
programs; and penetration, armor, and adaptive protection developments.
52

The ARL 6.1 research management team needs to acknowledge that true knowledge research has to
be allowed to fail often. Acceptance of the outcome as acceptable and expected for high-risk research is
an iterative process for research and is totally necessary. For example, one to two orders of magnitude
improvement in performance requires new approaches, and one needs to allow many of them time to fail
before they succeed. ARL management needs to be tolerant of such failures.
The battlefield injury mechanisms portfolio seems well connected to outside groups, although the
nature of collaborations was unclear. As one example, ARL’s focus on quantifying and translating animal
data to humans appears strongly warranted, necessary, and positive. ARL is probably a leader in this
aspect of the field and could represent one area for ARL to lead the external research community.
Recommendation: ARL should pursue activities aimed at impacting and organizing the larger
external community in the battlefield injury mechanisms area, taking a leadership position in
sectors of the injury mechanisms S&T area within the DOD and the larger academic
community as well as studies of football and other sports head injuries.
The directed energy program focuses on radio frequency (RF)-DE and laser-DE. ARL leadership and
research teams successfully implemented some of the recommendations of the previous ARLTAB report.
Specifically, the collection of presented projects demonstrated a coordinated strategy across the enterprise
for the laser-related DE work, with indications of much greater collaboration with the Navy and Air
Force. Internal to ARL, principal investigators from the directed energy sessions demonstrated greater
awareness of work done with threat warning and countermeasures. Also, the radio frequency work needs
vision and focus demonstrated with laser work in order to show continuity and measureable progress.
Recommendation: ARL should form even deeper and more frequent collaborations both
internal and external to ARL to foster rapid innovation with operational and contextual
relevance as well as to improve the quality of the directed energy programs.
In the penetration, armor, and adaptive protection area, ARL has begun to quantify uncertainties in
computational results. UQ was not part of the ARL technical initiative one or two years ago, and ARL
was criticized by the ARLTAB for the lack of UQ. ARL has responded, and has demonstrated a good
start, but there is a long way to go. It was unclear, however, what level of research is being conducted at
ARL, or the number of people involved.
Recommendation: ARL should continue an emphasis in uncertainty quantification (UQ).

Further, ARL should purse the integration of UQ into its data-to-decision workflow that
includes modeling and simulation, experimentation and design. To accelerate integration and
given the complexity of its objectives, ARL scientists and engineers should leverage software
and methodologies developed at other Department of Energy and Department of Defense
laboratories.
53

Information Sciences
The Panel on Information Science at the Army Research Laboratory (ARL) was charged with
reviewing ARL research in the broad areas of information sciences—network sciences, cyber sciences,
sensing and effecting (S&E), system intelligence and intelligent systems (SIIS), human and information
interaction (HII), and atmospheric sciences. A 2-year cycle of review has been adopted for this purpose
with the focus in 2017 on reviewing activities in SE, SIIS, HII, and atmospheric sciences.
ARL research in Information Sciences is focused on developing and enhancing science and
technology (S&T) capabilities that allow for the timely acquisition and use of high-quality information
and knowledge at the tactical edge, for both strategic operations planning and mission deployment.
Included in this approach are technological advances that support information acquisition, reasoning with
such information, and support to decision-making activities including collaborative communications. The
research is expected to generate new technology to both manage and effectively use information flows in
the battlespace of the future. Research in these areas falling under the broad categories of SE, SIIS, HII,
and atmospheric sciences were reviewed in June 7-9, 2017, at Adelphi, Maryland. This chapter provides
Research in sensing and effecting is organized around understanding of sensing capabilities and
exploiting information gained through sensing to drive effectors. Research in this area focuses on
understanding the sensed phenomena, including the effects of the environment on the phenomena. Both
sensing and effecting necessitate detailed understanding of corresponding physical behaviors that
generate and utilize data, an ability to extract relevant information from sensors of different modalities, an
understanding of the complex and variable environments in which these sensors operate, and an ability to
combine information from multiple modalities and other contextual and environmental information.
Research in effecting focuses on the understanding of transforming information into physical events, and
provides insights into the nature and characteristics of related phenomenology, thereby helping design
effectors that provide the desired physical effects. This work includes development of advanced
algorithms to improve signal-to-noise ratios for all sensing modalities. The overarching goal of sensing
and effecting is to deliver relevant information to decision-makers in a timely manner to facilitate
effective action.
System intelligence and intelligent systems research at ARL seeks to both understand and exploit
interactions between information and intelligent systems, such as software agents or robots. The research
portfolio includes a range of topics, from core machine learning, vision, and natural language
organization and understanding, to integration of information and decision making. The primary essential
research areas (ERAs) reviewed includes artificial intelligence and machine learning (AI and ML) and
human-agent teaming (HAT). An important consideration in this work is to think of information as data in
context, and to use this data to develop automated intelligence in perception, reasoning, planning, and
collaborative decision making, with applications in cyber virtual environments or decision support
systems. Research in this area complements efforts being undertaken in the Sciences for Maneuver
Campaign involving intelligent systems concepts applied to vehicles or robotic platforms.
The Army faces significant and growing challenges in the area of human and information interaction.
This is an extremely broad area with obvious linkages to human sciences and other information sciences
54

programs. The growing use of unmanned vehicles and robots and the expanding influence of social media
in planning and operations are creating a challenging communications environment for the Army of 2040.
The need for effective communications between humans and machines, often without the ability to
determine whether the communication is from and to a human or a software agent, introduces additional
degrees of complexity. The need to operate effectively in this environment is creating a need for a new
transdisciplinary science of HII. This is a crucial area for ARL basic research because many of the core
issues are deeply intertwined. Mission requirements such as the need for diverse teams to operate under
extreme time pressure in unpredictable, resource-constrained battlefield conditions are not a consideration
in most academic research. This is an area in which ARL has a strong potential, and an imperative, to lead
in research. Of course, as they do this, relevant academic work need to also be monitored and exploited.
The characterization of the battlespace environment and prediction of optimal conditions for engaging
an adversary with overwhelming force present significant challenges for the Army of the future. The
Army is likely to find itself engaged in a number of challenging environments, from complex terrain to
sprawling urban areas. There is a need, therefore, to characterize these diverse environments and develop
accurate, relevant, and timely predictions of their future state on spatial and temporal scales useful to
Army operations. The related need to collect and process accurate, relevant, and timely environmental
characterizations in austere conditions, and translate that data into actionable environmental intelligence
for field commanders will also pose new challenges to the computational sciences community. The
research being conducted by the Battlefield Environments Division is addressing these challenges, and
includes a mix of analytical, computational, and experimental projects. There are clear linkages of the
individual research projects to the nine ERAs considered essential to the success of Army operations in
the 2030-2050 time frame.
SENSING AND EFFECTING
Research projects in sensing and effecting covered thematic areas of nonimaging sensors (acoustic,
electric, magnetic, seismic), radar sensing and signal processing, image and video analytics, sensor and
data fusion, and machine learning. The nonimaging research presentations focused on electric and
magnetic field sensing and on acoustic classification. Radar sensing included advancements in landmine
and improvised explosive device (IED) detection, adaptive radar transmit-waveform selection, and radar
imaging in congested spectrum environments. Image and video understanding research included cross-
modal face recognition, real-time image object recognition, and human action learning and recognition.
Sensor fusion research included multimodal image fusion and understanding, combined text and video
analytics, and multimodal fusion for detection and estimation. Machine learning research was used as a
tool in several of the S&E projects.
The research portfolio contains a mix of basic and applied research, and some of the projects show
clear potential for transition into Army applications and products. Noteworthy programs include electric
and magnetic field sensing, research on the next-generation IED and landmine detection platform,
computational advances in electric field modeling, cross-modal face recognition, along with the
development and dissemination of a cross-modal face recognition data set to the academic research
community, and innovative approaches to fuse textual context with image features to improve machine
learning of human activity.
55

Nonimaging Sensors
This area of research included projects in modeling of electric and magnetic fields, electric and
magnetic field sensing technologies and exploitation algorithms, and acoustic classification algorithms.
The research on electric and magnetic field modeling represents high-quality algorithmic development,
and seeks to apply computer science expertise to an electromagnetics problem. The research program
contains a healthy mix of fundamental work related to understanding of phenomenology, sensor
development and assessment, and algorithm and application advancements. The work reviewed was of
high quality, and this group provides national leadership in this important area. Mentorship and
development of early-career researchers is especially strong, and it is clear that ARL is nurturing the next
generation of talent in this area.
Radar Sensing
Projects in this category included research in radar imaging and in adaptive transmit waveform
selection in congested radio frequency interference (RFI) environments, and on further development of
standoff landmine and IED detection. The research on radar imaging in RFI environments builds on a
well-established program at ARL, and has potential to transition into operational use. The research on
optimal transmit waveform selection in an RFI environment is in early stages, and is addressing an
important problem that will enable radar sensing as the RF spectrum becomes increasingly crowded and
complex. The research on a vehicle-mounted radar sensor for landmine and IED detection is advancing
the capability in multisensor design and phenomenology, and is addressing important technical challenges
needed for transition into operation.
Image and Video
Research projects in this grouping included cross-modal face recognition, real-time image object
recognition, and human action learning and recognition. The cross-modal face recognition research is
advancing the capability of facial recognition by comparing images collected in one modality with a
catalogue of facial images from another modality. Researchers collected and disseminated a data set to the
academic research community to facilitate development by others and grow collaborations with ARL.
Research on real-time image object recognition is in the early stage, and seeks to develop algorithms that
could be deployed on video sensors to automatically detect objects of interest from video streams.
Research is also being conducted to develop algorithms to model and to learn human actions and
activities from image sequences or video streams.
Sensor Fusion
These research projects included multimodal image fusion and understanding, combined text and
video analytics, and multimodal fusion for detection and estimation. The project on multimodal image
fusion and understanding applies algorithms to fuse point-cloud and other image data for use in scene
situational awareness. Advancements in improving robustness in fusion of data from coarse-grained and
fine-grained sensors, such as acoustics and imagery, were presented. Research attempting to improve
human activity classification using text and linguistic information is in an early stage of development. The
research on cross-modal face recognition is strong and the data collection and dissemination efforts are
commendable. The research on fusing point clouds with imagery was of good quality.
56

Machine Learning
This research included decentralized estimation and learning methods; in addition, machine learning
was used as a tool to support several of the projects described in other areas discussed earlier. The
distributed learning research was considered to be of high scientific quality. The demonstration was on a
“toy” problem. It will be useful to assess the research as it matures to the point where it can be
demonstrated on a relevant problem in an experimental test-bed.
While the overall quality of the research was considered as strong, it was not uniformly so across all
presentations. Some researchers did not demonstrate good understanding of the existing state of the art.
As a result, these presentations lacked a clear connection to existing research, and in some cases, it was
not clear why existing techniques were inadequate to address the problem at hand. Researchers were
generally able to articulate those aspects of their work that made it unique to Army needs and had Army
relevance. However, this was not uniformly so, and it is advisable that all researchers develop a strong
understanding of Army relevance and uniqueness of their work, as well as an ability to communicate this
in an effective manner. Such understanding would contribute to developing research problem statements
better aligned to needs, and consequently higher impact of successful research endeavors. Applied
research programs would benefit from clear transition planning that includes mission fit to a present or
envisioned Army need. Basic research programs would benefit from at least some early consideration of
this relevance.
The research staff at ARL is well qualified and equipped to take on problems of increased complexity
and challenge. To this extent, the research problem statements in some instances were less ambitious than
they could be. This was noted more often in cross-disciplinary endeavors. ARL has in its existing
workforce, either within the S&T portfolio or in the larger ARL cadre, the research talent and expertise to
formulate and pursue challenging research goals. Researchers pursuing such cross-disciplinary projects
are encouraged to establish collaborations and mentoring relationships with experts in allied areas early in
the problem formulation stage.
The posters, presentations, and oral summaries were generally good in providing sufficient details
about assumptions and experiments to assess the quality of the work. Some presentations and posters
lacked important detail about assumptions being made, and details about the experimental design setups
that led to the reported performance results. It was in some cases difficult to understand what experiment
was conducted and what significant observations or conclusions could be drawn from the results.
Research impact and understanding would be improved by additional attention to these descriptions in the
posters and oral presentations.
following.
Nonimaging Sensors
The relevance of the work to the ARL Key Campaign Initiatives (KCIs) and core campaign enablers
(CCEs) was not very clear, and Army relevance and impact was hard to gauge. The acoustic classification
research is considered as a collaborative project between the sensing and effecting and battlefield
environment groups. Tighter collaboration and integration with the battlefield environment researchers
would yield enhanced impact. The performance gains might be improved with a more integrated
consideration of propagation physics in the sparsity models.
57

Radar Sensing
The research on radar algorithms in spectrum-congested environments would benefit from a stronger
development of a transition strategy; this would help to focus the research to ensure addressing transition-
relevant research issues. Additionally, with the rapidly shrinking radio frequency (RF) spectrum available
to radar, the research would benefit by considering transmit waveform adaptation to effect power
notching as an approach to avoid interference with communication bands. The cognitive radar research
has opportunity for stronger impact if a more ambitious problem statement were adopted. In particular,
more general waveforms that use more of the available spectrum could be considered. This research could
consider sensing the available spectrum on both transmit and receive, and leveraging the cognitive radar
research conducted by Defense Advanced Research Projects (DARPA) and other Department of Defense
(DOD) service laboratories.
Image and Video
The research in real-time video analytics is at an early stage, and the object recognition aspects were
not completely clear. The research might benefit from a crisper and clear problem statement and
methodology approach, and from a stronger, more integrated consideration of object recognition. In
addition, a comparison to semi-supervised learning using convolutional neural networks could improve
the impact of the research. The research on human action recognition would benefit from a clearer
presentation of the mathematical construct, experiment design, and experimental results.
Sensor Fusion
The research on cross-modal face recognition would benefit from consideration of larger subject
database sizes, and also from the inclusion of battlefield environment atmospheric effects (such as dust or
smoke) on recognition performance. As regards the research on fusing point clouds with imagery, other
research groups, both within ARL and within the Night Vision Laboratory, are considering related
problems, and researchers are encouraged to understand and leverage other similar research in this area.
The research on improved multimodal fusion would benefit from a clearer articulation of the problem
statement, and by framing the research approach more clearly in the context of existing research literature
on the topic. It was difficult to assess how the proposed approach yielded the stated performance gains.
SYSTEM INTELLIGENCE AND INTELLIGENT SYSTEMS
SIIS research has produced key results in areas relevant to Army needs, including the understanding
and analysis of complex environments and streaming data, navigation, exploration, and mapping of the
physical world. The work on unsupervised learning of semantic labels in streaming data, and the
synergies between visual analysis and efficient exploration of environments is noteworthy. It has
identified gaps in the state of the art and is being disseminated in leading venues.
The SIIS research portfolio also includes work related to text analysis, language understanding and
dialogue, information integration, and decision making. The approach of collaborating with researchers
throughout ARL as well as on the outside to develop a continuum of work, ranging from information
analysis (in SIIS) to decision support (in HII), will yield good dividends. The research on integrating
event ontologies to increase the coverage of events provides near- to mid-term utility. In the longer-term,
approaches that combine curated resources and more data-driven approaches as well as integrating text
58

and video could provide added benefits. Research on natural language dialogue to support human-agent
interaction, and agent intervention for decision and information integration, is important in supporting
efficient communication and information analysis. Much of this work is focused on novel methods to
collect data to support the design of such systems, and results were preliminary but promising.
The sample of research presentations and posters reviewed in this cycle are in broad subject areas of
information understanding, information fusion, and computational intelligence. A summary of their
assessment follows.
Ontology for Event Understanding
A critical issue for analysts and decision makers is to rapidly and accurately identify events of
concern in documents. An important component of such an information extraction data pipeline is an
ontology, and the ongoing work explores representations that could be used to encode and integrate
previous ontologies to obtain better coverage. The research seeks to evaluate how the integration affects
information and decision tasks, and is of good quality.
Knowledge Representation and Modeling for Understanding Information Amalgamation
This work seeks to use knowledge representation to define a computational model of the value of
information (VoI). In previous research, the team developed a fuzzy logic-based approach to model a VoI
metric across various information characteristics (e.g., reliability of source and credibility of content) and
mission contexts. The current research extends this by investigating how analysts combine
complementary and contradictory information. Some potentially interesting trends were identified, but the
data is sparse and the result may be context specific. The underlying effort to use cognitive modeling to
assist in tactical intelligence data fusion has great midterm potential, and the linkage to experts in
cognitive modeling is laudable.
Toward Natural Dialogue with Robots: BOT Language
This project seeks to advance the state of the art in natural language dialogue processing to support
human-agent interaction in the context of a collaborative search-and-navigation setting. A novel “Wizard
of Oz” methodology was developed to collect a corpus of naturally occurring variation in language in this
setting. Initial results have helped in understanding of dialogue structures in these settings.
Air-Ground Robot Team Surveillance of Complex 3D Environments
The focus of this work is to improve autonomous surveillance via robots, with a human operator
helping to focus attention on relevant areas. Starting from the assumption of a known map, the robot
chooses a plan that is within a resource budget and maximizes coverage. The human operator can indicate
one or more regions of interest, and the robot adjusts its plan to prioritize these areas. This effort has
advanced beyond simulation, and the evaluation is being done with real robots in interesting
environments. Future work on this project includes the inclusion of additional constraints, including the
ability to select between ground or air vehicles.
59

Parsimonious Online Learning with Kernels
This high-quality work is focused on an important subset of learning techniques that are relevant to
Army needs. In particular, the research is focused on the “5 D’s” (dynamic, dirty, deceptive, dinky data),
and the approach is predicated on the use of online, parametric, kernel-based techniques. Results included
an evaluation for the proposed approach based on simulations, as well as some work toward a target
application predicting slip or oversteer based on images of the surface. The researcher was effective in
providing context for the work and identifying limitations.
Knowledge Representation and Modeling for Understanding Information Amalgamation
The objective of this work was to use knowledge representation to define a computational model of
the VoI. Previous research had focused on the development of a fuzzy logic-based approach to model a
VoI metric across various information characteristics (e.g., reliability of source and credibility of content)
and mission contexts. The current research extends this by investigating how analysts combine
complementary and contradictory information. Some potentially interesting trends were identified, but the
data is sparse and the result may be context specific. The underlying effort to use cognitive modeling to
assist in tactical intelligence data fusion has great midterm potential, and the linkage to experts in
cognitive modeling is to be applauded and supported.
Reasoning under Uncertainty via Subjective Logic Bayesian Networks
Inference is a challenging task in the presence of noisy, sparse, and untrustworthy data. This work
introduces subjective belief propagation, a new technique that extends belief propagation to efficiently
infer uncertain marginal probabilities over subjective Bayesian networks. This is a promising technique to
provide robust inference over uncertain relations.
Deductive and Analogical Reasoning on a Semantically Embedded Knowledge Graph
This research is focused on reasoning with brittle and incomplete information in knowledge bases,
and combines deductive and analogical reasoning and association by mapping information into a vector
space, and using the geometry of the space to find “related” paths. Previous work has looked at vector
relations in embedded knowledge representations, but none has combined this with deductive reasoning.
The ability to combine symbolic and continuous representations is a promising direction.
Unsupervised Semantic Scene Labeling for Streaming Data
The goal of this work is to augment visual perception by automatically extracting semantic groupings
of objects found in video. This project was particularly strong, both for Army relevance and in its
execution. In particular, the use of unsupervised techniques means that the approach does not require
tedious labeling, a difficult task for video. Furthermore, the approach is applicable to long-running video
without requiring excessive storage. To restrict the propagation of error introduced by unsupervised
learning, the approach incorporates an ensemble gained from training the model on a sliding window of
frames with periodic resets. The work compares favorably to previous techniques such as over- and
undersegmentation entropy and graph-based techniques. Results of this research have been published in
papers presented in top vision conferences.
60

Autonomous Mobile Information Collection Using Value of Information Enhanced Belief Approach
This research focuses on a framework for planning and optimization in multirobot environments. By
using a model based on partially observable Markov decision processes, and by using the VoI to help
prioritize, the approach is expected to pick better plans to simultaneously meet mission objectives and
maximize information about the environment. The research is at an early stage of development, and a
simple prototype simulation has been used in early investigations and described in a paper presented at
this year’s International Conference on Robotics and Automation (ICRA), a top-tier conference in
robotics.
A Crowd-Sourced Integration of Intervention Thresholds for Decision Agents
This research project seeks to develop an approach for supporting human decision making with
automated agents. A new methodology was developed in which a single-player game was developed to
help an analyst solve an information-based problem that was deployed on Amazon Mechanical Turk
(AMT) with hundreds of participants. The primary outcome seems to be building experience using
crowd-sourcing platforms for these experiments, and an initial indication that agents can help support
these decision-making tasks.
Robot Demonstration
A robot capable of autonomous navigation, exploration, and mapping was demonstrated. The robot
was tasked with mapping a complicated space via light detection and ranging (LIDAR), inertial, and
odometry-based sensors. The on-board algorithm was designed to select an optimal pathway to map the
interior, and was sufficiently capable to avoid dead-end paths that might impede its task. Additionally, a
user could draw a polygon representing a region of interest, and the robot would dynamically adapt its
plan to map the desired space. This demonstration was an effective illustration of several ARL technology
thrusts integrated in a single robotic platform. It was nice to see an end-to-end demonstration of the Army
technologies of high relevance working together. The laboratory space where the demonstration was
staged represents an effective resource for researchers from all over ARL, as well as academic open
campus partners. In particular, the support for reconfiguring the space, the ability to fly small unmanned
aerial vehicles (UAVs), global positioning system (GPS) denial, and calibrated measurement equipment
(e.g., the VICON camera system) are all features that make the space valuable for ongoing research.
It is evident that artificial intelligence and machine learning have been assigned priority as a crucial
area, and that ARL is organizing its research portfolio, particularly in SIIS, to address key gaps relevant to
the Army. There is specific emphasis in areas including learning with sparse and noisy data, learning in
adversarial settings, unsupervised learning, vision, decision making under uncertainty, and natural
language processing. The research projects are appropriately resourced. Bright early-career researchers
have been hired, and there appears to be a healthy cross-disciplinary integration. Overall, the facilities and
computational resources seem adequate. The facility conducting experiments with robots, UAVs, and
humans is especially impressive. This facility will help attract staff and visiting researchers to the
laboratory. Engagements with universities seem to be a great source of collaborations and new talent. It
will be important to continue to invest in this area, both internally and especially with university and
industrial partners, where the pace of development in this area is extremely rapid. ARL plans for
establishing the Intelligent Systems Center (ISC) to facilitate open campus collaboration and unification
61

of effort to take intelligent systems from tools to teammates are an indicator to grow and promote
partnerships.
There is some concern about researchers being isolated from each other, as well as the outside world.
There was some evidence that SIIS researchers within ARL were not always aware of similar or parallel
efforts. This is not unique to ARL, and all government laboratories generally can benefit from closer
interaction with industry and universities around Army-relevant needs. One way to further enhance this
awareness might be to create and share relevant data sets.
An opportunity exists for SIIS researchers to look for interesting science with applications of machine
learning in domains where the Army may have a great amount of labeled data, and perhaps even
structured data, such as logistics or medical records. There may be immediate gains in the application of
methods that have been successfully used by industry (e.g., deep neural nets). Engaging with such
applications will help keep SIIS researchers current on big data tools and methods developed elsewhere.
following.
Ontology for Event Understanding
In addition to evaluating the impact of extended coverage on an end-to-end pipeline task, an

important future direction would be to integrate curated ontologies with more data-driven approaches.
Moving from a controlled experimental setting to one with operational components would accelerate
data collection and research progress. In addition, it might be interesting to consider the integration of
language with richer multimodal signals (vision, mapping).
Reasoning under Uncertainty via Subjective Logic Bayesian Networks
Generalizing the work to realistic applications will require a number of advances, such as moving
from binary to multinomials, inference over directed acyclic graphs (as opposed to trees) and general
graphical models.
A Crowd-Sourced Integration of Intervention Thresholds for Decision Agents
The findings were preliminary, and would benefit from a more detailed statistical analysis of the
relationship between decision making and agent guidance, and a broader framing of the decision-making
options. The work has been published in defense-based sensing venues, and might benefit from a broader
AI or decision-making setting.
HUMAN AND INFORMATION INTERACTION
Human and information interaction (HII) is a new program in the Information Sciences Campaign,
and has been in operation for about one year, bringing together researchers from disparate disciplines and
technical backgrounds. “Interaction” is what distinguishes HII in the information science and human
62

science research space. It applies to the interaction of groups of individuals and agents with information,
as well as individual discovery, creation, and use of information. Research into interaction requires
expertise in collaboration, team dynamics, social influence and other social science subareas. Interaction
is at the center of battlefield dynamics, the expanding use of social networks for communication, and
information dissemination, and it enables the dismounted soldier to attain and maintain extensive
situation awareness. With a focus on the dismounted soldier, issues of land-based real-time interaction
rather than future forecasting are paramount. The objective of HII research at the ARL is to develop
models, methods, and understanding of data and information generated by humans and intelligent agents
in a complex, multi-genre network environment. It further examines tools to respond to user information
needs with due consideration of user variability and mission constraints, and thereby to develop timely
and accurate situational understanding.
Multimodal State Classification
The research addresses an important problem of human state classification. It focuses on the use of
machine learning to classify whether a user is in a threatened or challenged state based on wearable
sensor data. The research focus resides in comparing and contrasting multiple modalities and multiple
methods of classification, and provides breadth to the research while ensuring broader strategic relevance.
The potential of the research is high, and the investigator demonstrated understanding of how this work
fits into the broader scientific community.
Exploiting and Mitigating Cognitive Effects of Real-World Luminance Dynamics on Visual Search
This research explores the use of deep neural networks to predict human search behavior under
dynamic range stimuli for images and video by illuminating the complex patterns of interaction that
impact search. Specifically, the focus is on developing models of visual search behavior in dynamically
changing, high-luminance environments such as those available in high-definition resolution imagery and
videos. The work is in collaboration with the Human Research and Engineering Directorate at ARL.
Assessing Value of Information for Graph Understanding
The focus of the work is in developing metrics to measure the fitness of statistical graphs that better
inform the design of Army displays, and to develop automated techniques for generating task-dependent
statistical graphs. There exist some human-factors guidelines for generating graphs and charts, and the use
of these guidelines has been shown to yield improved performance. The work has clear alignment with
Army needs and represents collaborative work between ARL, the Naval Research Laboratory (NRL), and
the Institute for Human and Machine Cognition.
Social Computing: New Directions for Army Relevance
This is a new area of research and is focused on applications in decision making, in establishing
context, and in understanding or interaction. The scope of research tools and technologies is quite broad,
and will require input from many disciplines. The program is in a formulation stage, seeking to establish
appropriate research directions in social computing relevant to Army needs, and to coordinate disparate
efforts in the field across ARL. Machine learning and graph theoretic analysis have been identified as key
technologies to pursue in this context.
63

Intelligent Information Management for the Battlefield
This user-focused research is of high quality, has a clear articulation of the associated scientific and
technical challenges, has clear connections to Army needs, and presents a transition path and partner. The
researchers demonstrated strong understanding of what work is being done in the broader scientific field
and how their work fits into that framework. As such, the researchers know what community to address
and where to publish.
The work is creative, and the data and methods are appropriate to the objective. The researcher was
able to gather and make clear utilization of end-user feedback. A strength of this work is the use of live
experiments to gather data, shape the research objectives, and test the overall approach. The
demonstrations and live experiments provide a test-bed and framework for VoI research that lays the
groundwork for other related projects in the HII area. This work has the potential to develop context,
user-adaptive information management, and processing techniques that will support the dismounted
soldier in tactical environments.
Opinion Formation and Shifting
This project seeks to understand how people passively interact with social media, and will contribute
to a theory of information propagation through media-enhanced social networks. The researcher is very
talented and has good ideas. The collaboration within ARL is good. Using a set of experiments, the
investigator is seeking to derive empirical thresholds for opinion formation and shifting given diverse
types of media. This is clearly of Army relevance. The investigator has solid scientific credentials, is good
at designing experiments, and would be well served by mentors from the area of sociology or social
psychology of opinion formation.
Internet of Battlefield Things
The complex cyber physical environment that will characterize the battlefield of the future has
defined the Internet of Battlefield Things (IoBT). The vast amount of information available from
networked things in IoBT offers endless possibilities of using analytics to support military planning and
operations. This will require humans to interact with intelligent agents in a variety of ways. The research
project seeks to understand the dynamics of human-agent interaction, and to build models of such
interaction so as to simulate different battlefield scenarios to develop optimized strategies for dynamic
resource and task allocation across human and agents in IoBT. Toward this end, the approach is to
formulate an optimization problem based on a cross-layer (Com/Net, actuator, human layers)
mathematical model for optimal resource allocation. This project has clear Army applications and seeks
to collaborate with researchers in the NRL. Developing constraints reflective of human knowledge and
perception will certainly enrich the optimization problems but the approach for doing so was not
presented.
The HII agenda at ARL is very broad, and without a dramatically narrowed focus, it will be difficult
to make sustained progress that will transition to Army-relevant applications. A key challenge, therefore,
is to leverage the deep understanding of Army needs and requirements to narrow the focus and to reorient
new projects. In a number of the projects, the research questions are too broad, and are not designed to
lead to actionable and transitional results in the near term. In cases where research is linked to Army
needs, the specific Army issue is not clearly articulated in a way that drives the research question and the
64

methodology. The approaches used are often not well matched to the available data or to the research
question. A strong scientific mentoring program is needed to ensure that promising projects realize their
potential.
In response to suggestions from the 2015-2016 ARLTAB assessment, ARL has recruited two
psychologists into the HII program and is pursuing the recruitment of an economist. Although this
initiative is to be lauded, it is important to recognize that talent with necessary scientific expertise in the
areas of social, team, and group behavior and communication is needed to support the HII projects. Data
scientists and information scientists have been hired, further strengthening the computer science focus.
The research staff distribution continues to require additional balancing to bring greater understanding of
social theoretical underpinnings of the research.
HII has begun to identify synergies with other parts of the ARL, and may benefit from creating a
“social network model” of ARL, identifying those individuals whose work is synergistic with their own.
There is untapped potential to coordinate across areas concerned with the social side of human behavior,
and the interaction with other agents and information. The creation of a challenge problem could be a way
for focusing HII efforts. Although a challenge problem has not been identified, HII researchers are
engaged and active in discussing what one might be. Mentoring, particularly matching researchers to
others with scientific experience in the methods or theories of relevance, needs to be a priority for ARL.
From the work that was reviewed, it appears that there are three areas of research around which
multiple projects can be organized, and where closer ties among projects would be a force multiplier.
These areas are VoI, social media and diffusion, and agent-based modeling. There is a strong focus on
VoI, and this is the most mature of the concentration areas. Projects related to this area could develop and
adopt a common use case, share data, and develop common formulations of VoI. The HII research needs
to expand to recognize the impact of context. Whether the information is of a socio-cultural origin or
technology related, the VoI is contextual. Taking context into account, such as the physical climate and
social environment, is a major challenge and opportunity. A focus in this arena has the potential for
elevating HII as leaders in the SBP-BRiMS1 community.
The other two areas are broadly part of social computing. Social computing brings in research from
over 20 scientific disciplines, most of which are not represented in HII at ARL. Despite the commendable
step of organizing a workshop, the HII group is still at a beginning point on the learning curve in this area.
Social computing is an area that is growing exponentially. The HII group is beginning to embrace a small
sector of this community, and could task itself with monitoring the larger social computing field as it
develops. To stay actively engaged, the group could identify Army specific problems that it needs to
pursue, and invest in the personnel qualified to pursue the work.
Social media and diffusion of information is of interest to HII, specifically as it relates to the role of
social media in information diffusion and propagation of false information. This field is extensively
researched outside ARL, with a decade of research addressing issues that need to be incorporated into
current HII formulations. Social media presents both an opportunity and a challenge, and for HII to be a
meaningful contributor, there is a need to define a uniquely Army mission, identify a transition partner,
and understand the legal authorities that constrain Army activity in this space.
The projects making use of agent-based modeling have strong potential but need to be more tightly
focused on Army needs. The value of agent-based models (ABMs) is that they support prediction and
forecasting. Since HII is more focused on near-term analytics rather than forecasting, the aspect of ABM
that supports forecasting is not needed. Another value of ABMs is that they support theory development,
and can be used to fine-tune field experiments. This aspect of ABMs has strong potential for the HII
program. The ABM expertise within HII needs to be enhanced to achieve greater progress.
It is important to underscore some disconnects that may detract from the impact of HII research. First,
there is a disjuncture between work being done, the stated goals and objectives, and the perceived time-to-
1
SBP-BRiMS stands for Social Computing, Behavioral-Cultural Modeling, and Prediction and Behavior
Representation in Modeling and Simulation.
65

get-to-that-objective. As an example, HII goals include operation in a complex multi-genre context and
understanding information content. Much of the research, however, considered only simple or single-
genre contexts and ignored content. Another example relates to the time frame allocated for a research
task. Social dynamics is meant to be a “distant goal”—that is, one that will take more than 15 years.
However, some HII researchers working in that area anticipate success in 1-3 years. This would be
acceptable if the ongoing work was research needed to enable the longer term goal; that does not,
however, appear to be the case. Shifting to the research program organization, it is to be noted that the HII
program overview mentioned beliefs multiple times, but the specific work that is most closely associated
is focused on trust and VoI. Trust and VoI are not the same as beliefs, particularly as they have been
operationalized. Either the goals need to include trust or the research could be shifted to beliefs.
To accelerate progress in the crucial areas of team dynamics, collaboration, and social influence, the
ARL could invest in additional social scientists to balance its strengths in engineering and computer
science. Specifically, a mathematical sociologist and an experimental social psychologist could help
significantly with ongoing and proposed work. A systematic effort to survey this expanding area to
identify specific Army-relevant subtopics and research results, and the groups working in this area, could
also be a priority.
The HII group could develop one or two challenge problems. To support the challenge problem(s),
the HII could consider a unifying integrated test and evaluation platform that is relevant to that challenge
problem. This would support building toward strongly Army-relevant technology, and support the testing
and evaluation of basic work. For every project, the researchers could articulate how this supports the
deployed dismounted soldier, and be able to pinpoint the specific Army relevance, the Army issue, and
the research they can do that is not being done elsewhere.
As noted earlier, the scope of the research program in HII could be further narrowed and refined for
maximal impact. One possible area for focus is a framework that defines how a joint human-machine
system selects a level of data resolution for effective interaction. A key issue in this task is to understand
what role the human operator needs to play in this selection and what is best relegated to automation.
There is no theory or framework to address this problem in a complex dynamic environment with teams
of humans and autonomous systems. A good theoretical understanding of this problem will be important,
because in the long run, the Army will need to transfer data at different levels of resolution for cultural
information, social network information, and general social informatics. HII has the opportunity to
establish leadership in core scholarship in this arena.
A second area to consider is a concerted focus on problems involving teams of three or more
interacting agents (e.g., two humans and one artificial). Currently, several HII researchers are focusing on
problems related to one human and one artificial agent. Fundamental questions to be answered in larger
team interactions include what enables high performance, how much information overlap is needed, and
what kind of transactive memory and knowledge memory do they need to develop. These questions
assume increased importance as the team size is increased, or as team members come from different
services or from different countries. Additional scenarios need to be considered where information cannot
be shared with all team members, and where there is impact of these omissions on performance. Another
fundamental question to be addressed is how the mode of communication—video, image, voice, and so
on—impact team performance.
A third area for consideration deals with the issue of the commander’s intent, and how intent can be
transmitted so it does not get distorted. It would be important to understand what factors cause distortion,
and how coordination and interaction among soldiers help resolve ambiguities in their understanding of
the intent. Much research in HII could be refocused to address issues directly related to commander’s
intent. A platform could be built for testing and evaluating models and conducting experiments vis-à-vis
commander’s intent in a multi-genre complex information environment.
following.
66

Multimodal State Classification
The research could be strengthened, particularly with respect to methodology. There was lack of a
clear articulation of connection to an Army mission, and the associated lack of a clear transition path and
transition partner. This project could benefit from the combination of mentoring in the broader scientific
community, and advice from potential end-users and colleagues to help identify the specific Army focus
and transition path. While the initial results indicate success in working with physiological data, the
likelihood of it working with more cognitive or social factors is in question. The project could benefit
from using much larger training sets.
Exploiting and Mitigating Cognitive Effects of Real-World Luminance Dynamics on Visual Search
Initial results demonstrate feature interaction but no link of this to in-the-field utility was
demonstrated. It was unclear if there was sufficient data for model development and testing, especially in
the context of using deep neural networks. Another goal of the project is to develop a high-definition
resolution model of visual saliency, and it would be useful for the researchers to review the literature on
how saliency has been addressed in previous work.
Understanding Theoretical Information Interaction: The Development of a Standard Model Using

an Agent-Based Modeling Framework
The objective of this research is to address challenges in developing fundamental theories of human
information interaction. This project is central to HII, but needs to be better formulated. The project is
relatively new, and both the domain and the methodology are new to the investigators. While the
investigators are creative and energetic, the goals of the project are overly ambitious, and the team has
limited expertise in agent-based modeling. A project of this scope and the development of a credible
agent based model in this space could take years and would employ vast quantities of data for model
instantiation and testing.
There are many extant theories that are considered to be theories of human information interactions.
These include social influence theory, constructuralism, information bias, and so on. The presenter had
knowledge of a few of the behavioral theories such as confirmation bias. The approach being followed is
to develop an agent-based model where complex interactions can be evaluated. However, there are
multiple existing agent-based models that have been used to reason about human information interaction.
It is important for researchers to learn about these models, and that the model being developed be docked
against these existing models. A simple model in NetLogo is being developed that is great for learning
but will not scale well for large simulations. This model does not control for structural effects such as the
networks connection information, the social networks connecting people, the communication networks
over which people interact, and the knowledge networks of who knows what, or has access to what
information. The authors may consider employing a meta-network approach to look at high-dimensional
networks. The work could also benefit from the use of agent-based modeling tools that allow for network
analytics. Furthermore, the project could be strengthened by building on the measure of VoI developed
elsewhere in HII and using that as one of the dependent variables in the simulation. That would both
increase Army relevance and create synergies with other work in this area.
67

Assessing Value of Information for Graph Understanding
The project has a well laid out plan but is limited in that it does not have a comprehensive theory, or
empirical mapping of the relation between graph type, decision problem, and data. Without this mapping,
the overall guidelines developed are likely to be too generic.
Social Computing: New Directions for Army Relevance
The scope is somewhat narrow, and reflective of a strong computer science focus. Social computing
is a well-established area, involving many scientists and multiple conferences devoted to the subject. The
investigators present a description of social computing as dealing with social media but demonstrated
limited understanding of the breadth of work in that field, and a grasp of the key challenges.
Value of Information-Based Mission Needs Matching
The objective of this research project is to evaluate a set of metrics that could be used to assess value
of information within a simulation environment. This is a relatively small project that uses existing
metrics with clear Army relevance. The project could be expanded to other metrics and the simulator
expanded to other types of conditions. This, and indeed all the VoI projects, would be strengthened by
integrating all the simulation and test-bed environments, creating a comprehensive set of VoI metrics, and
utilizing consistent approaches to measuring and calculating them. Tighter integration across all the VoI
projects would be a force multiplier. The simulator and the metrics are really best for evaluating physical
systems, and as currently developed, will not work for sociocultural information. Indeed, such
sociocultural information is difficult for all the VoI projects. Furthermore, the current simulator is limited
in the kinds of missions that it can accurately reflect. The authors might consider developing a taxonomy
of mission types so as to accurately gauge where the current simulator is relevant, and where new
simulators are required. It is proposed that the work be extended to metric aggregation. This is a worthy
goal, and the researchers could build better awareness of aggregation techniques cited in the literature.
Modeling and Analysis of Uncertainty-Based False Information Propagation in Social Networks
This work proposes an opinion model based on subjective logic and then uses it to study how to
mitigate the impact of false information using counter-narratives. The model is mathematically well
specified but operates at a very abstract level. The problem is important, and the researchers need to get a
better understanding of other ongoing research related to fake news and rumor propagation. The work is
directed primarily at spread of false information through social networks, but the model does not take into
account known network effects. Further, the emphasis is on thinking of networks as graphs. The vast
literature on information movement, fake news opinion dynamics, and social change in networks that has
come out of the social sciences is very relevant, and needs be considered. The researchers also need to
dock their model against leading models in this area that are based on social influence. They would also
benefit by gaining familiarity with the broad literature on information diffusion, misinformation, and fake
news. Key factors not considered are the psychology of opinion formation, social influence,
constructuralism, network topology and its impact on diffusion, network externalities, information biases,
and confirmation biases. This is a highly interdisciplinary problem and requires expertise beyond the
discipline of engineering. The researcher is quite talented and would benefit from going to the more social
science conferences in this area—such as the INSNA meeting or SBP-BRiMS.
68

Opinion Formation and Shifting
The principal weakness of the project is a lack of consideration of a broad range of existing work on
opinion formation and shifting. Specifically, this work is classifying people based on number of messages
by type of media, with no attention to content, saliency of content, relation of content provider to the
person being influenced, and so on. Prior work has demonstrated that it is not solely the number of
messages that matter, but rather factors such as the affective relation between the sender and receiver, the
content of the message, the affective nature of the message, the inclusion of image/video/numbers, the
role of the sender in the receiver’s network, the strength and direction of the receiver’s current opinion.
Another issue arises from the claim of obtaining results using “content laden message examples,” but
content is explicitly removed in the experiment. This is clearly inconsistent. Use of AMT for subjects is
also problematic. It is not clear that the subjects were sufficiently tested for linguistic skills commensurate
with the project and for the learner type. Individuals have different ways of learning, and some are more
prone to learning from video and images than others. This needs to be controlled in the experiment.
ATMOSPHERIC SCIENCES
The research portfolio of the Battlefield Environments Division (BED) can essentially be divided into
two thrust areas that include improving environmental understanding of the planetary boundary layer
(PBL) and processes that operate on small spatial and temporal scales, and on developing appropriate
environmental intelligence tools for deployed soldiers to use in austere, complex operating environments.
The review in this cycle followed on the heels of an assessment held at White Sands Missile Range
(WSMR) in June 2016. Thus, this interim report spans a period that is one year instead of the two-year
cycle of reviews that was adopted in 2013-2014.
BED scientists have accomplished a great deal, and these achievements will be highlighted
throughout this section. Additionally, BED leadership provided specific, point-by-point responses to the
findings and recommendations from the 2015-2016 ARLTAB report, and greatly assisted the panel in its
work.
The research projects reviewed by the panel during this cycle included detection and characterization
of chemical aerosols, acoustic and infrasound sensing, development and fielding of a meteorological
sensor array at WSMR, and advances in small-scale atmospheric model development, verification, and
validation. In addition, updates were provided to the panel on past large-scale projects. These are
described shortly.
It is important to note that additional projects were also discussed with the panel. Significant progress
has been made in the past year with the automated aerosol Raman spectrometer project. The work is
commendable, and a recent paper2 is particularly useful in showing the utility of the project as well as
discussing the benefits of the BED work over other aerosol characterization systems. Additionally, work
that was completed through the Small Business Innovation Research (SBIR) program to improve drone
performance is also noteworthy. This innovative leveraging of the SBIR program has been successful in
developing enhanced drones that incorporate environmental turbulence data, thereby improving drone
movement. The SBIR work that has been undertaken by the BED has multiple civilian and DOD
2
D.C. Doughty and S.C. Hill, 2017, Automated aerosol Raman spectrometer for semi-continuous sampling of
atmospheric aerosol, Journal of Quantitative Spectroscopy & Radiative Transfer 188:103-117.
69

applications and, most importantly, has led to the development of additional, fundamental (6.1) scientific
research questions.
Design of Experiments for Verification and Assessment of Fine-Scale Atmospheric Forecasts
The presentation was essentially an update of the team’s activity over the last 12 months. Good
progress has been made regarding the development of an appropriate design of experiments (DoE) matrix
for initial testing. The matrix, developed for 40 weather research and forecasting (WRF) model
simulations, was recently completed and is ready for execution. The significant challenge of developing a
DoE approach for atmospheric numerical weather prediction (NWP) models, an approach that has never
been attempted, had been noted by ARLTAB earlier. The ARL team has done an admirable job of
surveying the scientific community to evaluate the project’s feasibility and establish a unique niche for
this work. Much work remains to be done, particularly with model verification. At present, the team
proposes to employ root-mean-square error analysis of the 40 model runs, with the intention of
performing more detailed verification using subdomains, and possibly leveraging the National Center for
Atmospheric Research (NCAR) method for object-based diagnostic evaluation (MODE), which is
currently being evaluated at ARL.
Multiscale Atmospheric Modeling for Tactical Army Nowcasting
The researchers have continued the performance evaluation of the WRF model for the May 2013
Moore, Oklahoma, tornado case. The model was tested in 2016 with a number of different data
assimilation schemes, including a hybrid of four-dimensional (4D) data assimilation and three-
dimensional (3D) variational data assimilation that is both computationally efficient and fairly accurate.
Researchers have been running a very-large-eddy-simulation (VLES) version of WRF over the Jornada
Experimental Range (JER) region north of WSMR, with a 300 m horizontal grid spacing and 130 vertical
levels. Preliminary results from the VLES WRF show great promise in the realistic reproduction of
detailed PBL vertical motion patterns over the JER region. Results from a version of WRF run on a 450
m horizontal grid over the Dugway Proving Grounds were also presented. This set of simulations sought
to evaluate combinations of PBL and surface physics parameterizations in order to determine which
combinations produce the most accurate representations of the PBL in complex terrain.
Vortex Filament Method in Microscale Atmospheric Modeling
Researchers have pursued three distinct approaches for the atmospheric boundary layer environment
(ABLE) model for predicting mean wind, temperature, moisture, and turbulence over urban and complex
terrain in near real time. These included computational fluid dynamics, the vortex filament method
(VFM), and the lattice Boltzmann method (LBM). There are two attractive characteristics for the VFM
that include its potential use as an engineering tool to help one understand the nature of complex terrain
and urban-scale atmospheric flows, and the validity of the VFM approach in high Reynolds number
scenarios. Preliminary results feature the ability of VFM to reproduce thermal bubble rise by comparing
simulations to theoretical results from Shapiro and Kanak (2002),3 and demonstration of isotropic
turbulence in a 3D periodic box.
3
A. Shapiro and K.M. Kanak, 2002, Vortex formation in ellipsoidal thermal bubbles, Journal of Atmospheric
Sciences 59(14):2253-2269.
70

Atmospheric Boundary Layer Environment Model Lattice Boltzmann Method
The LBM approach solves particle kinetic equations in lattice directions, and is well suited for
modeling turbulent flow with complex boundaries. It is a more applied method than the VFM discussed
previously, and has thus far satisfactorily reproduced theoretical results for stably stratified flow over a
mountain ridge, convective flow forced by a heated ground surface, and flows around buildings. It is
worthwhile to note that this project and the VFM have benefited greatly by collaboration with researchers
from multiple universities and other ARL directorates.
Visualizing Terrain in Augmented Reality
This project relates to 3D visualization of terrain and ground cover, and is illustrative of collaboration
between BED and other Information Sciences Campaign researchers. The terrain and ground cover data
were collected from UAVs and could be visualized using either 3D glasses with a projection screen or a
holo-lens wearable visor. Both methods were demonstrated in the poster presentation. ARL collaboration
on this project with the University of Southern California Institute for Creative Technologies is
noteworthy. The project has clear Army relevance, providing soldiers with the ability to plan and rehearse
dangerous missions in complex terrain and urban areas. This technology also has potential applications in
a training environment.
Infrasound Research and Applications
This more-than-15-year applied research (6.2) project is enhancing techniques for detecting blast
signals using infrasound waves—typically at frequencies below 20 Hz. These waves can travel for
hundreds of km horizontally and vertically, making them ideal for standoff detection of events such as
missile launches. The approach uses time-of-arrival (ToA) differences of sound, and incorporates these
differences with atmospheric properties to improve the models to estimate ToA. This work is critical to
the proper identification of activities and their locations. The research incorporates real-world
measurements of infrasound ToA from rocket launches from Wallops Island, Virginia, under varying
conditions; uses models with the appropriate parameters to simulate the actual systems; and then
compares the actual measurements with the modeling results. This project therefore incorporates an
excellent work plan to validate the utility of the models. One of the unique aspects of this project is the
relationship between ARL and the recipient of the technology, the Army’s 501st Military Intelligence
Brigade in the Republic of Korea. While the technique in its current state is not ready for field use by the
brigade’s soldiers, ARL is currently involved with the brigade in testing and evaluating the methods. A
clear transition plan to different parts of the Army is in place, and includes, among other aspects,
collaborations with international partners (e.g., Republic of Korea). The work is strong, and
improvements to the mathematical modeling (to improve the time-delay estimates) are possible through
collaboration and connection with researchers in the sensing & effecting group.
Acoustic Classification Algorithms for Complex Propagation Channels
This project uses the modeling of sparse data sets to model acoustic events with the intention of
developing event classifiers that can be used by soldiers in the field. Work was conducted at the basic
research (6.1) level and is in the final year of 6.1 funding; it will transition to applied research (6.2). The
work technically represents a joint battlefield environment effects/sensing and effecting project and its
science is solid.
71

Detection and Characterization of Chemical Aerosol Using Single-Particle Laser Trap Raman
Spectrometer
This project has successfully demonstrated a laser-based technique for isolating, detecting, and
identifying the chemical compositions of micron-size particles of multiple phases with unprecedented
speed and accuracy. The researchers’ method to capture and suspend the particle vastly reduces the
analysis time to the order of seconds, and allows a sampling frequency that is much superior to other
methods that are currently in use. The work is exceptional and novel, and has the ability to revolutionize
the aerosol science field as well as all industries and technologies that rely on aerosol science. The unique
feature of being able not only to measure composition but also to track changes in the size of the particle
paves the way for new investigations and applications, and is therefore transformational to the aerosol
field. As one example, the work may assist in developing basic information on the aging of particles,
thereby helping to determine their origin. This project has the potential to spawn additional applications
outside DOD, and once it has been adapted to an operational environment, provide soldiers in the field
with an additional level of protection from chemical weapons. Several papers on this work have been
published, are in the review process, or are being planned for submission to high-quality peer-reviewed
journals.
Meteorological Sensor Array
The deployment of the meteorological sensor array (MSA) at WSMR will enable unprecedented
continuous examination of atmospheric phenomena crucial to our understanding of atmospheric flows
over complex terrain at high horizontal resolution. In addition to fixed instrumentation, it also a mobile
component (LIDAR) and instrumented unmanned aircraft system (UAS). As a means of laying the
groundwork for this ambitious project, the ARL team has been very engaged with universities, industry,
and international agencies in field observation experiments such as mountain terrain atmospheric
modeling and observations (MATERHORN), and an experiment that is just concluding in Perdigão,
Portugal. The deployment of the MSA has four phases. Phase I is the experimental test-bed of 5 towers
located near the WSMR Las Cruces gate. These towers have been operating as needed for the past 2
years. Phase II involves the installation of 36 meteorological sensor towers at the Jornada Experimental
Range (JER) adjacent to WSMR, and a memorandum of understanding between ARL and the U.S.
Department of Agriculture (USDA), which operates the National Wind Erosion Research Network in the
JER, has been established. Phase II instrumentation deployment in USDA’s JER data collection network
is more than 50 percent complete. There are also plans for an acoustic array and instrumented UAS over
this area. Phase III will involve a dense instrumentation network installation on the west slope leading up
and over San Andres Peak at 2510 m elevation to complement the ARL/USDA network over JER. This
capability, when fielded, will provide detailed atmospheric information over this extremely mountainous
terrain, with resulting data certain to benefit the Army’s ability to operate in such challenging
environments. In Phase IV, ARL hopes to gain approval for installation and operation of instruments in
the High Energy Laser Systems Test Facility domain, which is part of WSMR. To date, nearly a dozen
agencies and universities have either provided equipment or are interested in collaborating with ARL on
this project. For instance, NRL is contributing soil moisture sensors and rain gauges to the MSA for
verifying satellite-based moisture measurements in a desert environment.
Opportunities and Challenges
Current staffing in the BED is 53, and includes 45 government personnel and additional contract
personnel including two Ph.D. senior researchers, two postdoctoral fellows, two computational support
staff, and two administrative support personnel. These numbers reflect a sharp decline in BED contract
72

personnel over the previous year, in both research and support staff. The engagement of postdoctoral
fellows has greatly benefited the quality of research within BED, and the decline in their number is
viewed with concern. The engagement of postdoctoral fellows has been shown to be effective in the past,
and needs to be continued, especially if promising candidates can be converted to permanent staff. A new
ARL initiative is the establishment of nine essential research areas (ERAs) that emphasize
interdisciplinary efforts considered essential to the success of Army operations in the 2030-2050 time
frame. BED researchers have been quick to show the linkages between their individual projects and the
appropriate ERA. BED can be integrated within additional multiple ERAs.
The opportunities and challenges to BED programs are interrelated. First, BED’s thrust areas touch
nearly every S&T campaign outlined in the Army Research Laboratory S&T Campaigns 2015-2035.
Although battlefield environment and weather are technically assigned to the sensing and effecting area
within the Information Sciences Campaign, the atmosphere and its effects can impact vehicle maneuver,
lethality and protection, human sciences, and materials science—all of which are campaign areas in the
ARL S&T plan. A related issue is the continuing challenge that BED faces to ensure that their unique
expertise is being leveraged by ARL research projects that may have an environmental sensitivity. Both
of these represent opportunities for additional research collaboration that can enhance the overall strength
of the ARL portfolio, and strengthen its position for transitioning useful technology to the field. On the
other hand, these interactions are also challenging because many project scientists outside BED are not
aware of the expertise residing in the Directorate, and may not even understand that atmospheric effects
and impacts need to be incorporated into their research. This latter point was illustrated in several projects
from the system intelligence and intelligent systems and sensing and effecting areas, where the
researchers either were not aware of the need to incorporate environmental sensitivities into their research
methods or felt that environment was unimportant or could be mitigated by other strategies.
BED has continued to leverage all available resources in advancing the research plans. In addition to
postdoctoral fellows, the Division has made extensive use of other avenues for assistance on their
projects. For example, students from Navajo Technical University are at ARL as summer visitors.
Another instrument that BED has utilized is the cooperative research and development agreement
(CRADA). Planned future CRADAs with NCAR and Vaisala Corporation will help all three BED
branches work more efficiently—with the Vaisala CRADA being particularly important as the MSA is
fielded. However, there are some inefficiencies regarding the length of time to get CRADAs established.
In an environment with limited resources, this presents a challenge to maintaining research at the leading
edge. BED continues to be challenged by steadily declining resources, but there is a strong scientific
imperative to incorporate BED expertise in multiple projects across the ARL portfolio. If full
collaboration potential with other related laboratory projects can be realized, fundamental science will be
advanced in highly significant and potentially groundbreaking ways.
following.
Multiscale Atmospheric Modeling for Tactical Army Nowcasting
Challenges here include a more complete understanding of objective verification statistics for radar
reflectivity, where the current equitable threat scores are somewhat counterintuitive. The preparation for
eventual use of data from the meteorological sensor array (MSA) being fielded at WSMR was also
discussed. The researcher commented about the potential of leveraging findings from the Design of
Experiments project in future research once that project has produced usable results.
73

Atmospheric Boundary Layer Environment Model Lattice Boltzmann Method
Future work includes improvements in turbulence model coupling as well as radiation model and
surface thermal model coupling, continued validation using laboratory and field observations, and
initialization with larger-scale model data and LIDAR observations.
Visualizing Terrain in Augmented Reality
The researcher was directed to a study published by Hembree et al. (1997)4 that incorporated cloud
scene simulation model output into a mission planning and rehearsal simulation tool called PowerScene.
This early work was limited by hardware computational capabilities, with the simulation tool losing its
real-time frame rate as a result of assimilating the weather model information. Significant advances in
computational power render the approach more tractable, with potential for realistic weather
incorporation into contemporary simulation tools.
Acoustic Classification Algorithms for Complex Propagation Channels
The departure of a key BED researcher on the project has resulted in the modeling effort not
sufficiently incorporating atmospheric effects in the physics of acoustic propagation. The current
approach is to mitigate or overcome the effects of the atmosphere through deployment of sensors in
multiple locations. However, the project could benefit from increased collaboration with other BED
subject matter experts for an improved integration with the atmospheric components.
The research portfolio in the Information Sciences Campaign reviewed in this current cycle was
expansive, and covered areas spanning sensing and effecting, system intelligence and intelligent systems,
human information interaction, and atmospheric sciences. The projects reviewed range from those
advancing fundamental science to those focused on enabling technologies and applications. The ongoing
projects demonstrated relevance to the future missions of the Army, and were generally of good technical
quality. There are additional opportunities to further drive scientific innovation through enhanced
integration and collaboration across campaigns.
ARL has focused on increasing the number of Ph.D. scientists on the research staff in critical areas of
expertise and this has had a measurable positive impact on the overall quality of technical work. The
nature of the research in information sciences dictates a workforce with significant technical diversity,
including strengths in the social and mathematical sciences. This diversity and added strength in the social
sciences is also critical to the HII initiative. Among the ARL researchers, there was generally a good
awareness of external research and connections to professional organizations and external research
communities; research results are appearing in respected conference proceedings and in archival journals.
There is room for even broader dissemination of these results to a larger scientific community. As noted
in earlier reviews, the mission-oriented thrust helps differentiate the ongoing research from efforts
pursued elsewhere, and creates opportunities for impactful technical contributions. The impact of the
work can be further enhanced by clear articulation of unique, cutting-edge research questions.
4
L. Hembree, S. Brand, W.C. Mayse, M. Cianciolo, and B. Soderberg, 1997, Incorporation of a cloud
simulation into a flight mission rehearsal system: prototype demonstration, Bulletin of the American Meteorological
Society 78:815-822.
74

The work in sensing and effecting was assessed to be generally of high scientific research quality, but
not uniformly so. The research portfolio represents an appropriate balance of theoretical and experimental
work, and many of the more mature programs, but not all, show a transition into practice or use by other
areas. The research generally reflected a good understanding of the problems being considered,
appropriate statement of the problem being pursued, knowledge of the appropriate methodologies to
address the problem, and a good knowledge of the state of the art and the relevant research pursued
elsewhere. In many cases, the researchers are able to articulate Army relevance and unique aspects to
Army needs when that was the case. The researchers are academically well qualified to carry out the
research problems that they are pursuing. Some of the work is being published in top venues, but
researchers could be given additional guidance and encouragement to present their work at leading
conferences, and then pursue publication in top archival journals. The increase in postdoctoral researchers
and early-career researchers is a positive trend. The quality of the early-career ARL researchers in the
S&E area is impressive. The ARL process of using co-ops and internships to recruit a strong cadre of
emerging researchers appears to be yielding positive results. The facilities are adequate to the needs of the
researchers and of the research, including access to computational facilities and to laboratory
instrumentation.
The overall research program in SIIS is aligned with the current and future needs of the Army. It is
clear that the researchers within SIIS are generally conversant with modern techniques and approaches
from the rapid advances in machine-learning research. SIIS researchers were forthcoming and articulate
in their presentations and responses to questions. Thorough materials were provided well in advance of
the meeting, newer publications were provided, and the research team was available for clarification after
its initial presentations. The presentations of the different projects were of somewhat variable quality. In
part, this was due to the limited scope of some of the projects, or that the project was in an early stage and
did not have a comprehensive set of results. In some instances, the presentations simply failed to highlight
key questions, methods, and interim or final results. For some of the more experimental work, it was not
clear whether the researchers were building an engineering artifact to develop initial insights and validate
approaches, or whether they had a more scientific paradigm that started with a hypothesis that could be
crisply stated, and that was the basis for the design of the experiment. In many instances, the research
results are being disseminated in top conferences or journals, but not uniformly so. While not every piece
of good work will be published in the most highly regarded venues, the site of publication is a good
indication of high-quality work and provides excellent visibility for the researcher and ARL. Furthermore,
publishing in high-level journals will attract the top academia researchers (and their students) to problems
most relevant to the ARL mission.
The research focus in the area of HII is relatively new, and includes many projects that are at an early
stage of inception. As a result, the technical quality of the work is mixed. The projects were highly
variable in the extent to which they had identified the relevant computational and social theories. For the
nascent research projects, there is the need to develop a stronger Army research focus, and to use this
focus to narrow the scope of work as a path to realizing meaningful results. The more mature work has
strong Army relevance but tends to focus on single human-technology interaction. While important for
demonstration purposes, the impact of the research could be enhanced by reformulating the problems to
include scenarios involving Army teams interacting with information and technology. The researchers are
clearly skilled and show enthusiasm and the desire to grow and to learn. Early-career researchers could
use more mentoring, and all researchers would benefit from better ties to the broader scientific
community. The field is highly interdisciplinary, and research productivity and impact would be enhanced
if the HII team was better balanced to include additional social scientists. The work being done by HII
researchers is being disseminated at relevant conferences, including multiple papers at leading venues
such as the SBP-BRiMS conference.
In work related to atmospheric sciences, the overall scientific quality of the work is very good, and
quite comparable (and, in a few cases superior) to research conducted at successful university,
government, and industry laboratories. Researchers are very familiar with the underlying science and
cognizant of research being done elsewhere; in many cases, they have been either dialoguing or
75

collaborating with researchers outside ARL. In all cases, the researchers are aware of the potential
challenges associated with their projects. BED uses the use-inspired research5 approach, resulting in
researchers from different disciplines being brought into their projects. In one case, the dialogue led to the
recent staff additions of a mathematician from another ARL division and a National Aeronautics and
Space Administration (NASA)-supported scientist, which have greatly benefited the vortex filament and
lattice Boltzmann projects. In several other instances, collaboration and consultation with researchers
outside atmospheric sciences have enhanced project quality and applicability to the Army’s mission.
These types of approaches are commendable and need to be continued by the BED leadership.
The research portfolio in information sciences reviewed in this current cycle was expansive, and
covered areas spanning sensing and effecting, system intelligence and intelligent systems, human
information interaction, and battlefield effects/atmospheric sciences. The projects reviewed range from
those advancing fundamental science to those focused on enabling technologies and applications. The
ongoing projects demonstrated relevance to the future missions of the Army, and were generally of good
technical quality. There are additional opportunities to further drive scientific innovation through
enhanced integration and collaboration across campaigns.
The work in S&E was assessed to be generally of high scientific quality, but not uniformly so. There
was a balance between theoretical and experimental work, and in many of the more mature programs,
there was evidence of transition into practice or use by other areas. The research generally reflected a
good understanding of the problems being considered. The researchers are academically well qualified to
carry out the research problems that they are pursuing, and some of the work is being published in top
venues; more, however, can be done in terms of mentoring and encouragement to seek the most highly
rated venues for disseminating their results. The quality of the early-career ARL researchers in the S&E
area is impressive, and the capacity to successfully address much more challenging research problems
now exists within ARL. There is currently a mismatch between the quality of the researchers and the
degree of difficulty of problems that are being pursued.
Recommendation: ARL should actively encourage researchers to take on research problems of

greater complexity and scope.
The research in SIIS is appropriately aligned with the emergent priority accorded to the fields of
artificial intelligence and machine learning, and is seeking to address key gaps relevant to the Army.
Several early-career researchers have been recruited and have the appropriate background and academic
training to pursue the research goals that have been identified. The research projects are appropriately
resourced, and both the facilities and computational resources are adequate for the proposed work. There
is a healthy collaboration among researchers, both within ARL and with the external community.
Continued investments in this regard are encouraged, as in some instances it appeared that SIIS
researchers within ARL were not always aware of similar or parallel efforts elsewhere.
Recommendation: ARL should bring about greater understanding of available technical

expertise in SIIS-related areas across ARL, and build synergy across campaign thrusts to
leverage this technical talent.
5
A commentary on use-inspired basic research, “Thinking Beyond the ‘Quadrant,’” by Dr. F. Fleming Crim,
NSF Assistant Director for Mathematical and Physical Sciences,
https://www.nsf.gov/mps/perspectives/quadrant_august2014.jsp (accessed August 10, 2017).
76

HII is a small program that seeks to address an enormous area, with a wide range of potential
applications. HII would be well served by developing more within-group synergy and concentrating on
those aspects that are highly relevant to Army missions and the deployed dismounted soldier. Given the
nascent nature of the program, it is advisable to develop a strong mentoring program, perhaps using a
combination of external and internal mentors. There is also a need for developing a broader set of external
collaborators, including mathematical sociologists and social psychologists in the area of teaming. In this
context, it would be helpful both to send scientists to leading conferences in the area and to organize
internal seminars and workshops in advanced areas such as ABMs, information diffusion and the spread
of misinformation, and social media analytics. Last, it is noted that the work in HII is somewhat hampered
by not having access to all state-of-the-art tools and software. This concern could be alleviated by making
tools such as Repast and ORA and various image processing software available to researchers.
Recommendation: The HII research program is in a formative state, and ARL should develop a
strong mentoring program for the research team involving external and internal mentors. Also,
ARL should establish a broader set of external collaborations to accelerate progress of the
research agenda.
With respect to ongoing research in BED, the importance of the natural environment needs to be
featured more prominently in strategic planning documents in order to advance the fundamental science
while encouraging collaboration and dialogue between researchers whose projects have an environmental
sensitivity, and subject matter experts (SMEs) from BED. The exclusion of environmental effects,
whether inadvertent or intentional, could possibly jeopardize the successful fielding of promising new
technologies to the Army and the broader society. Environmental phenomena are crosscutting and impact
nearly every Army operation.
Recommendation: The environmental impact on Army operations should be given crosscutting,

prominent visibility in planning documents such as the S&T Campaign plan and the essential
research areas (ERAs) that are being formalized by ARL senior leadership. ARL should
evaluate the potential for environmental integration at the initiation of a new project, to enable
teams to incorporate multiple components into their approach at the inception of the design
process, and potentially enable faster basic science and applied technology development.
77

Computational Sciences
The Panel on Computational Sciences at the Army Research Laboratory (ARL) conducted its review
of ARL’s advanced computing architectures, data-intensive sciences, and predictive sciences at Aberdeen,
Maryland, on June 12-13, 2017. This chapter provides an evaluation of that work.
ADVANCED COMPUTING ARCHITECTURES
The advanced computing architectures group has evolved from operating and managing high-
performance computing (HPC) systems to serve the processing needs of the broader Department of
Defense (DOD) community to performing research focusing on tactical HPC at the edge. This places
innovative new systems at the points of need to support the soldier in complex operational environments.
The group is continuing to advance this evolution by responding to the recommendations of the 2015-
2016 ARLTAB report1 and ARL strategic priorities as defined by the ARL essential research areas
(ERAs). The group emphasizes research on innovative systems that match SWAPT (size, weight, power,
and time) and SWAPTN (SWAPT plus network) constraints to support workflows to enable “compute:
decide faster!”
The research portfolio comprises projects for evaluating and advancing new and emerging
architectures to enable artificial intelligence and machine language (AI and ML) computing with
extremely low size, weight, and power (SWAP) and real-time, large-scale data analytics at energy-
constrained points of need. There are also projects on models for quantum and classical networks for
secure communications. Many early-career researchers are advancing these areas, publishing in high-
quality venues and being recognized through best paper awards. ARL is complimented for trying to look
forward a good 30 years and thinking about the consequences of Moore’s law hitting a final plateau, and
also for thinking about performance-portable programming models for uncertain future hardware.
The group has made significant progress in evaluating the role of neuromorphic computing using
IBM’s TrueNorth processor and its leaky “integrate and fire” framework to enable high-fidelity
computation using many low-precision elements, and thus very low energy. TrueNorth uses simulated
leaky integrate and fire units to simulate neural computing. A key highlight is the demonstrated
capability to perform symbolic processing on neuromorphic processors, providing computing capabilities
equivalent to von Neumann processors at 1 percent of the power required.
1
Research Laboratory, The National Academies Press, Washington, D.C.
78

This is a great example of successfully addressing the recommendations in the 2015-2016 ARLTAB
report. Highlights include the fact that the project has successfully developed an end-to-end application
that uses deep learning to identify information in an image, and then processing that information using a
rules-based table look-up to determine optimal action. The team has also established a strong
collaboration with the IBM TrueNorth team and has emerged as one of the leading players in the
TrueNorth ecosystem with a respectable set of publications. The next step is to prototype a solution for an
important Army problem as soon as possible.
Hybrid distribution shared memory application programming interfaces (APIs) and C++ templates
were developed with the Adapteva Epiphany processor to demonstrate programmability to address
Adapteva’s multicore, shared memory Epiphany processor with a 2D array of compute nodes connected
by a low latency mesh network-on-chip. Army needs of inexpensive, low-power, and distributed
computing by leveraging open hardware and software. The team has also shown technical leadership by
having ARL achieve voting status in the OpenSHMEM community. This is important because
OpenSHMEM is an effort to create a specification for a standardized application programming interface
(API) for parallel programming in the Partitioned Global Address Space.
To serve the long-term Army needs for quantum secure communication and networking, measurable
progress has been made toward understanding the potential range of uses of quantum computing and
networking through modeling and simulation. A highlight is the work, in collaboration with
experimentalists, to demonstrate microwave-to-optical frequency conversion to connect superconductive
quantum logic circuits to optical communication.
Other notable results include the work on quantum walks to simulate physical systems. This is a good
example of a theoretical project, with two publications in 2017 in the prestigious Quantum Information
Processing journal. A complementary project is the one on quantum control for quantum-enhanced
distributed sensing. It has developed a practical tool (NIDE) to make it easier to design and evaluate
networks that can realize quantum-enhanced sensing. This project is important for keeping the Army
abreast of the latest relevant work on quantum networks. Moreover, the project’s methodology of
evaluation through simulation is a nice example of leveraging the HPC expertise at ARL.
In advanced computing architectures, the Computational Sciences Campaign has identified the goals
of advancing neuromorphic computing, many-core, co-processor, and ASIC-integrated architectures for
data analytics and tactical HPC delivered to points of need, while meeting strict SWAPTN constraints. If
a leadership role is the objective, then these goals are extremely ambitious. The space of design choices is
extremely large at the nexus of these types of new architectures, and their match to the data analytic, ML,
AI, scientific, and numerical workloads of Army relevance is not clear.
It would be advisable to consider a strategic approach that is grounded on a few critical end-to-end
processing workflows that have been determined as representative of the gaps that needs to be covered for
the success of the ARL science and technology (S&T) campaigns. Requirements emanating by these end-
to-end workflows could in turn provide a roadmap for the end-to-end system designs for research and
development (R&D). A portfolio planning approach could then be used to determine and prioritize the
projects to be advanced.
The projects presented as part of advanced computing architectures need to be positioned within a
larger framework so that they are motivated by the driving application or scenario to provide a complete
picture. This is the key challenge for this area. Within this broader context, there are many opportunities
to strengthen current projects including those described in the following. ARL could use the driving
applications to determine the key kernels to be used for evaluating new architectures, such as the
Epiphany processor, and to advance system development as opposed to using generic community kernels
such as those for linear algebra, Graph 500, and so on. The work on neuromorphic computing with the
TrueNorth architecture is commendable, but there needs to be at least another processor to enable more
79

generalizable findings. The project on control of quantum entanglement in cascade networks has credible
results. However, the presentation combined two or three disparate ideas that were not clearly delineated,
in that the linear and star topologies were for completely different applications. The Epiphany evaluation
is promising, but input/output (I/O) and full system design considerations are essential for successfully
accommodating large streaming data analytics workloads (which are not to be conflated with those for AI
and ML).
It will also be a useful exercise to consider some of these driving applications in a future context. For
example, this might include one in which a large central server trains an ML application, broadcasts a
trained model to small units in the field, monitors their output, and iteratively retrains the model on the fly
based on reported results. What would those small units look like? What would the server look like? How
will they communicate? What are the appropriate data formats and representations? What are the
appropriate provisions for tolerating various failures? Would ML training be the only function for that
server, or are more conventional “scientific HPC” calculations also relevant in the field in this scenario?
Among the projects presented, the following comments are noteworthy.
Programmability
This work described a framework for organizing communication among parallel processes for large-
scale HPC applications. ARL studied multiple approaches, including threaded message passing interface
(MPI), OpenSHMEM, hybrid OpenSHMEM/OpenCL, and distributed shared memory (DSM). ARL has
put the OpenSHMEM project on GitHub and is now a member of a standardization forum.
Templated Metaprogramming
This work described a software stack based on C++ templates for array-based programming. There
are optimizers tuned for various Intel architectures. However, it was unclear whether this is entirely in the
library templates, or whether some of it required digging into the C++ compiler. ARL researchers have
considered extending this to graphics processing unit (GPU) targets, but agreed that that may be a more
difficult problem because GPU architectures typically have multiple memories, not just a cached memory
hierarchy, and it is not yet clear how best to divide responsibility for memory allocation between compiler
and programmer.
Both the programmability and templated metaprogramming activities represent small projects that
seem to be approaching maturity and deployment; indications were that they would extend only to 2017
or 2018, and that seems appropriate. They are “internal tooling” projects, well worth the doing, that will
enable easier programming and improved portability for future software projects.
Opportunities for Computational Overmatch
This work considered hardware and software opportunities and challenges, attempting to look
forward 30 years and considering the consequences of Moore’s law hitting a final plateau. It also
considered performance-portable programming models for uncertain future hardware. The team
highlighted the need for large numbers of inexpensive, low-power devices that will be deployed in a
distributed fashion. This contrasts with designs being studied by the Naval Research Laboratory (NRL)
and the Air Force Research Laboratory (AFRL), where the Navy and Air Force are more likely to deploy
a smaller number of higher powered devices that perform centralized computation.
Portions of the work concentrated on the Epiphany architecture. Although it is lower power than
some other contemporary alternatives, it is designed for the construction of larger, more centralized
systems. Work on the Epiphany architecture and associated software concentrated on computation
80

(floating-point [FP] operations and template meta-programming) and internal communication (mesh
network, OpenSHMEM, and improving communication by exploiting writes in preference to reads), but it
entirely omitted any discussion or analysis of I/O capabilities and requirements (latency and throughput)
at either the hardware or software level. Putting a “petaflop in a cubic meter” on the battlefield to perform
centralized processing may well be worthwhile, but the work failed to put this in the proper context. ARL
needs to sketch a more complete application scenario to explore such requirements, even as a back-of-the-
envelope computation.
As an example, imagine deploying 10,000 sensors on the battlefield (maybe based on neuromorphic
processors). Sketch some idea, however rough, of how these sensors will communicate their data to a
central server, how that data will be fed into the server, what value will be added by applying petascale
computing to this data, and how results will be delivered and for what purpose. This scenario assessment
might lend some insight into whether a specific Epiphany-based design would have the necessary I/O
capability and capacity, and whether the software stack provides the necessary support for that capability
and capacity.
Part of the work also discussed the Army’s role as a fast follower of the Department of Energy (DOE)
in high-performance computing. As global competition accelerates and the United States faces budgetary
challenges, it is important to track global leaders. If not DOE, is it the Chinese? Is it Amazon or Google?
In this context, ARL appropriately mentioned the Google tensor processing unit (TPU).
The specific Google TPU design may or may not be the best architecture for an Army application, but
its designers paid very careful attention to overall application requirements and the context in which the
TPU would be deployed. Examination of Google’s recent white paper about the version 1 TPU shows that
Google paid as much attention, or more, to the motion of data into and out of the chip as to internal data
motion and computation. In the end, Google designed a systolic matrix multiplier with appropriately sized
memory buffers so that data can constantly stream into and out of the chip as computation is performed.
Another lesson from very recent developments in high-performance chip design: both Google (in
their version 2 TPU, announced only in 2017) and NIVIDIA (in their most recent GPU design, with
“tensor units” included) appear to have concluded that the sweet spot for machine-learning applications
(or a least training) may be 16-bit floating-point (FP) numbers rather than 32-bit or 64-bit. This is very
much in contrast to scientific applications. Indeed, the version 1 Google TPU supports 8-bit integer
arithmetic, and it may well be that moving to 16-bit FP for version 2 is actually a slight compromise on
performance in exchange for portability and ease of programming.
A plausible conjecture is that 5 years from now, the machine-learning community will have
“standardized” on a computational structure of 16-bit FP activation values, 8- or 16-bit weights (or
possibly 16-bit FP weights stored using some compression scheme and expanded on the fly), multiply-
add operations that multiply 16-bit FP values into a 32-bit FP accumulator, and a selection of nonlinear
operations that take 32-bit accumulations back into new 16-bit activation values.
Quantum Technologies
This work touched on “unique properties” of quantum physics—namely, coherence and

entanglement—but failed to give any real sense of how those unique properties enable the proposed
applications relative to classical approaches. How do these unique properties make quantum networking
better than classical networking? How do they contribute to secure communications and sensing?
DATA-INTENSIVE SCIENCES
The data-intensive sciences program presented its goals and progress in applied machine learning
(ML), neuromorphic computing, and cooperative multiagent control using deep reinforcement learning.
The data-intensive sciences overview and the two presentations on neuromorphic processing and
81

cooperative reinforcement learning were excellent. The data-intensive team has made a strong start in the
new research thrust in machine learning. ARL has not only hired new talent but also leveraged its existing
researchers. This is important because the ARL researchers bring additional scientific and application
expertise that will distinguish ARL’s research in machine learning.
Following up on recommendations from the 2015-2016 ARLTAB report, ARL has established an
advanced computing laboratory. It includes IBM’s TrueNorth brain-inspired computing platform, the
Adapteva Epiphany Network-on-Chip (NoC) and other emerging architectures. TrueNorth uses simulated
leaky integrate and fire units to simulate neural computing, and is considered exciting for two reasons.
First, the brain uses neurons that “fire” action potentials or “spikes.” Thus, TrueNorth can be seen as
coming closer to real brain computations than many other computing platforms. Second, TrueNorth uses
power only when “firing” events occur, making it a platform that uses very little power when it computes.
Exciting progress has occurred using the TrueNorth platform in two different ways. One approach has
been to write computationally transparent computer algorithms that exploit the TrueNorth platform (a so-
called top-down approach). This approach has been used to show how well-understood computational
problems can be solved in novel ways using timing differences between the occurrences of spikes,
including an award-winning paper that implemented a sorting algorithm that sorts lists in linear time on a
neuromorphic computing platform. An award-winning integer factorization paper was also published that
detailed the development and implementation or the algorithm on a neuromorphic computing platform.
The second approach has port-trained neural networks to the TrueNorth platform, allowing
demonstrations that previously learned computations can be carried out on a platform that consumes very
little power. The laboratory’s efforts in neuromorphic computing have blossomed since the panel’s last
visit into a significant and highly collaborative research effort.
In the Army High-Performance Computing Research Center, exciting progress has also been made
using deep reinforcement learning methods for multiagent control. Stanford University researchers,
funded through a cooperative agreement with ARL, have successfully extended the multiagent setting
methods previously used to train a single agent using cooperative reinforcement learning to achieve state-
of-the-art performance in several applications. The illustrated applications indicate the power of deep
reinforcement learning to address mission-relevant tasks dependent on multiple artificial agents.
ARL plans to port an instance of one of these applications to TrueNorth. It promises to provide a low-
power realization of a trained model. This is significant promising work in a new problem area in
machine learning. This could serve as a model for ARL’s internal research. The challenge is to build on
this work within ARL and exploit it.
Several projects are worthy of particular mention, as discussed in the following. The keystroke
biometrics project has made an impressive showing, winning multiple awards in an international machine
learning competition establishing ARL’s visibility in this research area. Similarly, the impact of machine
learning on the Army project has made a good step in articulating Army-relevant machine learning
challenges. The machine learning for planetary gearbox analysis work demonstrated a proactive approach
to preventative maintenance by automated generation of features. This approach could provide an AI-
based preventative maintenance technology—evidently a tremendous advance for Army operations. ARL
could benefit by exploiting theoretically based parameterized mathematical and statistical representation
methods for automated feature generation. The large-scale data analytics project, which was presented 2
years ago, has now entered the transition phase and is rebasing the software using current commercial off-
the-shelf (COTS) technology. Each of the three data-intensive computing projects (high-throughput
electrolyte modeling, discovering and quantifying atomistic defects in large data sets for assessing nano-
crystalline aluminum, and computational technologies for the reduction of highly nonlinear and
multiscale solid mechanics and structural dynamics models) are exploiting the same ARL-developed
software architecture for distributed simulation. Each of them represents different machine learning
82

challenges in detail. They all demonstrate successful machine-learned approximations applied to

simulation pioneered by ARL. Last, several other predictive simulation projects (Distributed Multiscale
Computation for Modeling Energetics, Multiscale Transport Modeling of Optical Response in GaN,
Development and Application of Multiscale and Multiresolution Models to Build Better Fuel Cell
Membrane Electrolytes and Modeling and Design of Macromolecules at Multiple Scales) demonstrated
this same approach. ARL could consider writing a book on its approach.
ARL has an outstanding opportunity to continue making progress exploiting and extending recent
developments in machine learning. Two opportunities identified in the 2015-2016 review remain exciting
avenues for further development. The first is use of ARL’s expertise and resources in high-performance
computing as a home for large mission-relevant data sets. The second is exploiting the potential of
neuromorphic computing platforms to allow the advances in deep learning to be implemented effectively
in light, low-power devices for deployment in the field.
The first of these opportunities remains to be more fully exploited; this agenda lies at the heart of the
data-intensive science mission, and needs to be more vigorously pursued. Application of machine
learning to Army problems is target rich and just beginning. The challenge ARL faces is how to expand
the initial efforts to applications beyond the excellent application speeding simulation with learned
models.
As noted in the preceding section, the ARL has stepped forward in the area of neuromorphic
computing, and the efforts to date are applauded. That said, it is important to be aware that neuromorphic
computing is a diverse field of engineering whose potential is only beginning to be explored. Relying on
the leaky integrate and fire model of the neuron is one important approach, but far from the only one, and
it is not clear that it is ultimately advantageous from a computational point of view to make use of spiking
neurons.
Today’s deep learning methods rely on the use of continuous functions to allow very deep continuous
computational paths to be subject to gradient descent—their power depends on their differentiability,
whereas a “spiking” event is discrete and not differentiable. It is an important basic science question to
understand how the human brain succeeds in learning with spiking neurons, but from an AI/ML/robotics
point of view, solving this problem, or relying on any particular characteristic of real neurons, may or
may not turn out to be helpful. For these reasons, alternatives to the existing TrueNorth integrate and fire
platform needs to be pursued.
Neuromorphic computing appears to be one of the best approaches to low-power computing for
sensory data. The challenge is for the research not to be tied too closely to a particular neuromorphic
architecture (e.g., the leaky integrate and fire model for neurons). ARL needs to explore other low-power
realizations of neuromorphic computing.
Another opportunity exists for dramatically improving predictive modeling capabilities by
augmenting traditional scientific simulation paradigms with machine learning. Such a physics-guided
machine learning paradigm has the potential to bring the power of state-of-the-art machine learning
approaches to predictive modeling while leveraging the wealth of domain knowledge that is critically
needed for solving such problems. This line of research is beginning to be followed in a number of
projects (under predictive modeling) and presents a major opportunity for the ARL to take leadership in
this area that will be of great relevance to the computational science community at large.
Given the mission-critical nature of ARL applications, it is important to understand the limitations of
the machine learning paradigm. For example, typical machine learning methods do not produce a
statistical representation of the sample data that would enable uncertainty quantification, sample
generation for validation, and statistical fusion of different samples.
Again, a previous recommendation that ARL make a significant effort to become the data archiving
center for the Army is reiterated. Because the Army generates large volumes of data that are currently
83

isolated in local computer systems, the value cannot be extracted. A central repository is fundamental to
data-intensive computing. Such a repository will open research avenues and form the basis for
cooperation with academia. For example, the data sets from large-scale Army test and evaluation
exercises represent a unique resource and opportunity.
The panel recognizes the difficulty of hiring top-notch talent in machine learning. As a mitigating
strategy, ARL needs to place a heavier emphasis than usual on university collaboration with machine
learning experts and their students. ARL has a very rich and challenging set of real-world problems that
provide exciting challenges to machine learning researchers within ARL and elsewhere. To achieve this,
ARL needs to develop a specific collaborative process whereby university collaborators actually interact
with and transfer expertise to ARL personnel. ARL has a very rich problem space to drive its research—
but to execute it ARL needs top-notch researchers and collaborators and ARL needs to invest in its own
people and by promoting collaborations.
ARL could design and sponsor a machine learning competition based on Army-collected data under
deception scenarios. This introduces a new problem that will shape future research in machine learning
and computer science. Also, ARL could, along with its counterpart laboratories, advocate creating a
Defense Advanced Research Projects Agency (DARPA) program in the areas of robust methods in
AI/ML and submilliwatt software and hardware computing architectures.
PREDICTIVE SCIENCES
Predictive science has long served as a foundational component of the research and development
(R&D) practices of the ARL. From the early computational predictions of ballistic trajectories during
World War II to current work on advanced armor to ensure soldier protection, predictive computational
techniques have been an integral part of ARL’s execution of its national-security mission.
Current predictive science work at ARL focuses on developing the various “multi-” capabilities that
are required for accurate computational analysis, which include multiscale, multidisciplinary, and
multifidelity analysis. In each of these regimes, ARL researchers have a stated goal of including relevant
verification and validation (V&V) and uncertainty quantification (UQ) capabilities in their computational
analyses. A related R&D goal is to leverage results of prior computations toward developing various
surrogate reduced-order models that retain the accuracy characteristics of the original models while
requiring significantly lower computational time. Memory space, power consumption, and
weight/physical size are also other important metrics.
Under the current crosscutting schema of research campaigns and essential research areas (ERAs),
two specific ERAs underlie the key predictive science efforts presented at the review. The first ERA is
based on application of ML and AI and has shown substantial improvements since the 2015 ARLTAB
review. However, overconfidence in this arena is cautioned, as the 2015 review recognized a dearth of
relevant R&D in this field. Nonetheless, the improvements shown in this ERA (discussed in more detail
in the “Data-Intensive Sciences” section of this chapter) are laudable and demonstrate that in predictive
sciences there is a path to effective integration of simulation-based techniques with ML approaches. This
leads to a hybrid modeling methodology whose fast-running models (FRMs) can be used by soldiers in
the field to obtain results with sufficient accuracy.
The second ERA is motivated by a desire to better understand the physics of failure at a range of
scales, including the material, mechanical, and structural regimes. This ERA builds on long-standing
R&D programs at ARL for assessing proposed soldier-protection technologies ranging from assessment
of personal protection materials to design of advanced armor for vehicles.
84

There is a general improvement in the quality of predictive science R&D from 2015, and in
particular, a decreased variance in quality across the range of efforts presented. The notion of “relevance”
was apparent in much of the R&D presented, ranging from “relevant UQ” to “multiscale analysis relevant
to ARL’s mission.” This improved focus on mission relevance is laudable, so future panels can evaluate
R&D activities through this mission-focused frame of reference. Another area of improvement is in
recruitment and retraining of staff into roles with greater mission relevance.
Some specific examples of R&D results presented warrant extra attention, as they demonstrate an
appropriate focus on ARL’s mission sensibilities. The research on dynamic surrogate model evaluation in
a computational framework for scale bridging with application to multiscale modeling of RDX explosive
exhibited many noteworthy aspects of predictive computational science, including a scalable software
foundation utilizing the hierarchical multiscale (HMS) technology demonstrated during the 2015
ARLTAB review. The R&D work on improving interscale data communication, achieved by relieving a
bottleneck that would limit performance, provided impressive optimization results that demonstrate ARL
R&D leadership in advanced computing. This R&D effort also demonstrated the use of surrogate models
to improve performance, so a true cross-campaign activity resulted that demonstrated the
“multidisciplinary” aspect of ARL’s predictive science roadmap.
The vortex filaments work is a small-scale, exploratory research effort that included several
noteworthy features, including an academic collaboration and a focus on a spatial scale of especial
relevance to Army soldiers in battlefield conditions. Where much of atmospheric science R&D has
concentrated on the mesoscale (i.e., severe storms such as tornados), microscale effects often dominate
the practical utility of troop activities (e.g., deploying a drone in a hilly landscape where turbulent
vortices are produced by the interaction of wind and terrain). This project fills an R&D gap that is
relevant to ARL’s mission via a low-cost, high-impact-potential activity.
The integrated computational materials engineering for polycrystalline materials research effort was
an excellent demonstration of the practical value of multiscale analysis as applied to the design of new
materials, and it included several noteworthy goals. In particular, the project execution plan provided key
data science outcomes, including a curated data repository for both experimental and computational
results, and has initiated collaboration with the Georgia Tech faculty group working at the interface of
material science and data science. This project’s goals were mentioned as a means to effect the cultural
change required to bridge the intellectual gap between experimental and computational subject-matter
experts, which has been a long-standing impediment to broad adoption of data science methods.
There is continuing improvement of peer-reviewed publication record, with over 30 journal
publications. ARL needs to maintain this trend and work to increase the ratio of publications to staff.
Two specific cultural challenges were apparent during the review of the predictive science R&D
portfolio, along with one administrative challenge mentioned by several of the project personnel. The first
challenge is that the cultural change required for broad acceptance of predictive modeling and simulation
may not yet be in place, and it is not yet clear whether there has been a widespread adoption of the V&V
and UQ technologies required for computational analyses to be demonstrated as truly “predictive.”
Current ARL computational analysis practice does not yet promote standardized UQ processes, and V&V
activities were notable not by demonstration, but by more commonly being omitted in discussions of
R&D project activities.
This lack of acceptance of emerging methods for demonstrating the predictive effectiveness of
computational simulation was specifically called out in the 2015 ARLTAB review. UQ and V&V
processes need to become ubiquitous in future computational analyses at ARL.
85

A second cultural challenge is apparent at the interface of the two ERAs that help define ARL’s
Computational Sciences Campaign research foci. While there is considerable potential for effective R&D
that fuses predictive science results from ARL supercomputers with ML-based FRMs that would aid
soldiers in the field, not much evidence of this cross-ERA fusion was apparent from the presentations and
posters. This is a research venue that holds considerable relevance for ARL’s mission. The low-power
devices proposed for use in the battlefield will not be capable of performing high-fidelity predictive
computational simulations, so simpler models will be needed, and these models can be developed at the
interface of these two foundational ERAs.
The third challenge is an administrative one, and a common feature of federal R&D administration.
Successful exploratory (6.1) research efforts require a clear path for transition to practice, so that
exploratory efforts that are not successful (and by definition, exploratory research portfolios need to
include high-risk, high-return efforts that may not succeed) can be pruned from the research funding base.
Similarly, projects that demonstrate utility for ARL’s missions need to have a well-defined path to 6.2/6.3
reduction-to-practice funding so that they might ultimately lead to successful deployment under
operationally realistic circumstances. This challenge lies less with researchers and more with management
at ARL, but overcoming the challenge requires clear lines of communications from both parties.
Note that each of these challenges represents only one side of the larger challenge, with the other
representing a concomitant opportunity for improved R&D project relevance. These opportunities could
be pursued by ARL. ARL could extend new work in developing and deploying data repositories. In
particular, ARL is well positioned to serve as the Army’s data hub. One way to achieve this long-term
goal is to support and extend smaller scale efforts at creating curated community data archives of
relevance to ARL’s mission. Some of the R&D effort presented utilized prototype data repositories that
could be extended to support a transition path to the more ambitious goal of ARL providing the Army’s
data hub. ARL could reform a comprehensive gap analysis of ARL predictive science efforts, toward the
goal of identifying areas where collaborations are required to achieve successful R&D outcomes. This
analysis could support the longer term activity of performing research outreach efforts to academia,
industry, and other federal R&D institutions so that research outcomes are not compromised by omission
of key components needed for successful deployment. ARL could provide more technical detail on the
interfaces between predictive science and other R&D venues that support, or are supported by, this
foundational research area. For one example, the interface between the ML/AI ERA and the physics of
failure ERA could be better articulated (if it is already well defined) or better developed (if it is not).
As with any broad-based research and development effort, there is tension between the depth (focus)
and the breadth (coverage) of exploration, convolved with a balance between basic research and outcome
driven activities. Overall, ARL has made substantial progress in addressing its mission objectives since
the last review.
Many early-career researchers are exploring advanced computing architectures, publishing in high-
quality venues and being recognized through best paper awards. Kudos for trying to look forward a good
30 years and thinking about the consequences of Moore’s law hitting a final plateau, and also for thinking
about performance-portable programming models for uncertain future hardware.
The data-intensive sciences overview and the two presentations on neuromorphic processing and
cooperative reinforcement learning were excellent. The overall technical quality of the work at the early
stages of the research efforts represents a solid first step toward significant research contributions. The
data-intensive team has made a strong start in the new research thrust in machine learning. ARL has not
only hired new talent but also leveraged its existing researchers. This is important because the ARL
researchers bring additional scientific and application expertise that will distinguish ARL’s research in
machine learning.
86

The predictive sciences represent a good combination of machine learning within large simulations to
optimize multiscale model computations with the hierarchical multiscale (HMS) work. This is a
promising and important direction that would be important to continue into the future. It is encouraging to
see the fruits of ARL’s concerted effort in the past several years to develop a general-purpose framework
for HMS, and to see that the research is transitioning toward incorporating into HMS the lessons learned
from applying this technique to solve real problems, including HPC optimizations and the use of ML
methods. A note of caution is worth mentioning regarding the “past several years” HMS characterization
earlier. While the integration of the HMS work into new intellectual venues is commendable, the panel
noted that these R&D improvements are incremental additions to the HMS technology, and that new
starts for exploratory research are warranted to provide similar support for future ARL R&D.
The ARL team gains practical and intellectual advantage via thoughtful collaboration. ARL needs to
focus on identifying collaborators and points of leverage. ARL described a tripartite “lead, collaborate, or
follow model” for assessing research and development opportunities. ARL needs to create and share
white papers on the state of the art and to identify national thought leaders whose research, insights and
ideas can inform and guide research and development.
Conceptual diagrams of future battlefields can inform engineering implementation by mapping
activities onto the most critical problems and needs. ARL needs to convert these conceptual views to
tangible work plans, emphasizing those focused activities likely to maximize return on investment.
Last, ARL needs to think carefully about metrics for project success and define associated project exit
strategies, including transition out of this organization to achieve Army impact. In particular, the panel
emphasizes the need for transition funding and engagement for technology application and uptake.
Recommendation: ARL should develop more formal roadmaps and strategies for
higher-level research activities such as the ERAs. These plans should be less conceptual
and more oriented to feasibility, so that appropriate resources (financial and personnel)
can be allocated to research efforts, and so that impediments to success of these R&D
projects can be identified and remedied. One key component of these strategic planning
documents should be the development of transition policies for successful 6.1 projects
that permit continued funding for practical deployment. Also, ARL should maintain an
effort to track out-of-the-box concepts that could prove to be of interest to ARL. In
particular, ARL should focus on those that would have crosscutting benefits to more
than one of the three areas in the Computational Sciences Campaign.
Because the Army generates large volumes of data that are currently isolated in local computer
systems, the value cannot be extracted. A central repository is fundamental to data-intensive computing.
Such a repository will open research avenues and form the basis for cooperation with academia. For
example, the data sets from large-scale Army test and evaluation exercises represent a unique resource
and opportunity.
Recommendation: ARL should become the data archiving center for the Army.
The panel recognizes the difficulty of hiring top-notch talent in machine learning. ARL has a very
rich and challenging set of real-world problems that provide exciting challenges to machine learning
researchers within ARL and elsewhere.
Recommendation: ARL should place a heavier emphasis than usual on university collaboration
with machine learning experts and their students. ARL should pursue hiring of top-notch
87

researchers and promote collaborations with others. ARL should develop a specific
collaborative process whereby university collaborators actually interact with and transfer
expertise to ARL personnel.
88

Sciences for Maneuver
The Panel on Mechanical Science and Engineering at the Army Research Laboratory (ARL)
conducted its review of ARL’s vehicle intelligence (VI) programs—intelligence and control, machine-
human interaction, and perception—at Aberdeen, Maryland, on July 18-20, 2017. This chapter provides
INTELLIGENCE AND CONTROL
Among the priorities that the intelligence and control effort supports are artificial intelligence and
autonomy, robotics and autonomous systems, autonomous and intelligent ground systems to extend
warfighter reach, collaborative and intelligent air systems with improved maneuverability, and
autonomous and intelligent agents to achieve mission command and intelligence operations at all
echelons.
Work done under intelligence and control addresses gaps in the area of learning in complex data,
including artificial intelligence (AI) and machine learning (ML) with small samples, dirty data, and high
clutter; AI and ML with highly heterogeneous data; and adversarial AI and ML in contested and deceptive
environments.
The work presented to the review panel addressed issues in pattern recognition and segmentation in
video streams, linking perception and cognition, development, and implementation to world models, and
learning of terrain characteristics for improved robot navigation.
Overall, the efforts attempt to demonstrate (over a 5-7 year time frame) the operation of a
heterogeneous team of largely autonomous robots, both on the ground and in air, providing a “security
bubble” around a dismounted team of soldiers in military operations in urban terrain setting.
Based on that scenario, many of the studied technical tasks have immediate relevance. These address
perception that adapts to the environment, controls that learn from perception how to move on that terrain,
planning that considers threats, and architectures that integrate world knowledge with perception.
Other parts of the work are more basic. These address certain issues that are less likely to be tackled
by academic researchers—issues such as development of world models and linking perception with
cognition.
Novel neural nets or perception algorithms, for instance, may help in a wide variety of tasks if they
prove successful. The cognitive architecture work is of much longer range. It is unlikely that it will be
integrated in the 5-7 year time frame; nevertheless, it is important to properly address longer term
objectives and anticipated warfighter battle scenarios. As such, it is important that the Army continue to
pay attention to such issues.
89

The research group included several early-career individuals who earned their Ph.D. degrees during
the last decade or are currently working toward the degree. In general, the researchers exhibited
substantial familiarity with the literature, with current and future needs, and with relevant existing
techniques. Furthermore, the researchers were aware of the importance of presenting their work in
professional conferences and publishing it in refereed journals. Evidence was presented of collaboration
with academic researchers of significant relevant experience.
Four projects presented to the panel demonstrated a multifaceted strong effort in the area of
automated and joint human-robot path planning. These efforts are autonomous mobile information
collection using value of information-enhanced belief approach, context-driven visual search in complex
environments, air-ground team surveillance for complex three-dimensional (3D) environments,and
unsupervised semantic scene labeling for streaming data. These four projects complement each other
well. They offer an opportunity to develop and use joint performance benchmarks and to compare
performance and complexity of different approaches and algorithms on similar benchmark scenarios.
This work has made significant progress over the past 2 years—now fully unsupervised, with a
rational basis for selecting the parameters, and good results on merging nonadjacent blobs that belong to
the same class. The results exceed the state of the art. The method processes single images and streams of
image data. The researcher is developing important unsupervised capabilities.
World Model
This is a relatively small effort, trying to address a relatively big problem. World models for robots
have gone through phases for many decades. The four-dimensional (4D) RCS model was proposed some
three decades ago, before current robot capabilities had been developed. It is a good idea to once again
think through the needs of a robot world model, and which elements could be made explicit in a central
database versus distributed or kept implicit. This work is still in its infancy, and it is hard to predict where
it will have its greatest impact going forward.
Cognitive Robotics: Linking Perception and Cognition
The cognitive model used here, ACT-R, has a wealth of data connecting it to human cognition. That
makes it a reasonable basis on which to build a cognitive robot. This is a many-year effort. Over the past
2 years, this effort has made good progress on showing the generality of the underlying framework for
relevant robotics tasks in perception and reasoning. The next technical tasks proposed are importing
graphical data structures and doing probabilistic reasoning. If those capabilities can be integrated, that
will enable more advances. The overall cognitive vision is exciting and important.
Online Learning for Robust Navigation
This project tries to learn control parameters for a tracked skid-steer vehicle, where the parameters are
a function of different surfaces as inferred by an inexpensive onboard vision system. The system
demonstrates a learning approach that measurably improves the estimate of future paths. The researcher is
using a (disturbance estimation) technique that has worked well for many application areas. The system
and the approach certainly show promise; it is important to quantify its capability.
90

Autonomous Mobile Information Collection Using a Value of Information-Enriched Belief

Approach
This project uses value of information (VoI) as a central organizing principle for robot planning. The
VoI can be used to guide robot exploration (where it should go, where it should point its sensors). The
same principle can be used to infer value judgements made by human operators. The basic framework
seems to be sound, and can tie together work of several other projects. It will be important to find out
what are the actual performance advantages and limitations once this is fully connected to real robot
perception and mobility modules in a realistic environment and mission. The Partially Observable
Markov Decision Process (POMDP)-based reward function (capturing mission-specific human
knowledge) approach taken by this researcher has potential. POMDPs have been proven to be successful
in many application arenas.
Deductive, Analogical, and Associative Reasoning in a Semantic Vector Space
This work is performing deductive and analogical reasoning in a 300-dimension vector space, where
the location in that space is decided by data mining in a large corpus. Words are then connected to other
words throughout that high-dimension space. This gives a statistical basis for semantic queries, such as
looking for analogous concepts. The semantic vector space approach taken by this researcher has been
found to be useful in other domains.
This system combines human hints and automatic planning to provide time-constrained coverage
plans for a mobile robot doing surveillance. This project is addressing a very interesting problem, and is
just at its beginning. This system will give soldiers a forward air and ground surveillance capability.
Moreover, it will permit the injection of human knowledge to enhance route planning. This could help
address the mathematical intractability of the NP-hard 3D surveillance task. Work was evaluated using a
cluttered 3D urban environment. This is an excellent piece of work offering great potential and
opportunities.
It appears that all efforts can benefit from referral to benchmarks and test scenarios, and that the
development of common benchmarks and test problems would benefit the group (and researchers in
parallel efforts). A technique for detection or classification is never going to be “universal” in the sense
that the scene has characteristics that affect the definition of the problem and that the detection or
estimation or classification objectives are problem-dependent. The lack of common benchmarks and test
scenarios make it difficult to assess the usefulness of the presented algorithms and solutions, and prevents
meaningful technical exchange between researchers. Moreover, such benchmarks would significantly
assist the group in meeting precise short-term objectives and planning for longer term goals.
A related issue is scenario relevance. A number of the reviewed studies could benefit from real or
simulated data that are designed intentionally to be close to the scenarios anticipated by the warfighter.
Development of such realistic “battle” scenarios would highlight challenges that may not be adequately
addressed in the general literature, thereby increasing the value of the work to customers.
Another related issue is the complexity of the studied images. Detection of a movement in a single
dark silhouette of a dancer against a white background is a very different problem from detecting hiding
snipers in a noisy gray-scale video stream of a large cluttered urban scene during combat. It may be useful
91

to seek a common understanding of complexity for a library of images and video clips that can then be
used jointly to assess the performance of algorithms, techniques, and associated speed-complexity trade-
offs. Such characterization of the difficulty of the detection or classification or estimation problem in the
context of the expected applications is important to assess progress of the research over time.
The problems attacked by the intelligence and control research group require that solutions address
the following characteristics of the developed algorithms (as well as the algorithms to which they are
compared)—complexity, scalability, robustness, and operation in noise. Most studies presented included
some of these elements in the analysis, but their inclusion needs to be routine and systematic in all studies
where they are applicable. An issue that the group could consider is quantifying complexity. Some
problems will be solved as a consequence of technological (computing) progress (and it is therefore of
interest to quantify the time frame). Other problems and algorithms are inherently of intractable
complexity, and would therefore require either polynomial-time approximations or injection of side
information (such as aid by human intelligence) in order to make the problem mathematically tractable.
The following additional opportunities can be pursued to help the group’s researchers, management,
and ARL leadership. ARL needs to develop a more precise statement of group goals (long- and short-
term) and customers. The group and its members can benefit by articulating what will be realistically
achievable within 2, 5, and 10 years. Systematic benchmark studies can greatly assist with this type of
technology projection and planning.
Whenever applicable, studied techniques and algorithms could be presented along with an estimate of
their complexity and scalability. It needs to be clear if the studied techniques can benefit from ongoing
progress in computing technology (e.g., when their complexity is described by low-order polynomials) or
if they still require low-complexity approximations or side information in order to become practical.
Whenever applicable, studied techniques and algorithms could address robustness to both small-scale
changes and drifts and to larger scale failures and structural changes. Operation of algorithms and
techniques could include a report on their nominal performance as well as on their performance in the
presence of noise and interference (including the use of and development of realistic models of such noise
and interference).
The general tendency in robotics research is to require that physical implementation become part of
the presentation of new techniques and algorithms. While this approach has sometimes been criticized as
thwarting theoretical research, it is increasingly recognized for providing an often much needed “reality
check.” ARL owns and maintains a number of robotic platforms and, to the extent practicable, these need
to be used to test and demonstrate new approaches and new proposed methods. Moreover, testing on real
robots is likely to reveal practical challenges for which new theory is needed. Work with physical
platforms can also allow researchers to address fundamental hardware and performance limitations.
Discussion of techniques and algorithms could state fundamental limitations associated with the
approach and methods being taken. This discussion can be very illuminating to the researchers and
management, as well as future panels. Moreover, it can lead to substantive directions for future research.
ARL could consider pursuit of the following areas of research. Physical interactions with the
environment other than planned manipulation—pushing, sliding, kicking, running into, and other forms of
manipulation not using end effectors can often be useful. Robot models of human behavior—a robot
needs to understand the mental model of its human teammates—may answer the following questions:
What can the human see? How busy is the human? What is the human trying to do? While human
modeling is difficult, it is of great importance to properly address that critical soldier-robot trust factor.
The development of benchmark human-robot interaction/mutual-awareness models can be particularly
useful. Interplay between robot motion and autonomy and the communications infrastructure—robots can
communicate implicitly by their behaviors, or they can maneuver to enable chains of communications,
including line-of-site communications that are harder to jam or intercept. A small fleet of robots can, for
example, maneuver so as to maximize communications connectivity subject to interception and threat
constraints.
92

This research could benefit by addressing the following: How and what will agglomerative clustering
methods be integrated into? How can supervision and expert knowledge be systematically incorporated?
What benchmarks could be used to guide future developments? What is the plan for transitioning to the
Tank Automotive Research, Development, and Engineering Center (TARDEC)?
Additional questions on this research that when answered will lead to more progress on this important
area might include the following: How can the work be generalized to segment objects that do not have a
homogeneous appearance? It does a good job of handing objects with variance, but an object with disjoint
classes (e.g., a Dalmatian dog, with black spots on a white background) will not be properly segmented as
a distinct object—could the work be extended to pick one or more of the feature descriptors to ignore?
What is an example use for this technique? If one of the potential uses, for example, is segmenting a dirt
road from the surroundings, then that application suggests some performance criteria (speed, accuracy,
reliability) and a demonstration scenario.
World Model
A world model will be very useful for future collaboration activities. The relationship of this work to
future CRA projects and to other architecture projects needs to be clarified. Collaboration with others is
needed.
Cognitive Robotics: Linking Perception and Cognition
The researcher and team could benefit from more precise near-term goals and a 5-year benchmark. It
would be helpful to have a roadmap to show what additional work needs to be accomplished in this task
before it becomes useful for a real robot demonstration scenario.
Online Learning for Robust Navigation
It will be important to do the following going forward: Implement a state-of-the-art control system as
a benchmark, to see if the learning system really provides a performance advantage. Perform tests on a
wider variety of surfaces to measure the effectiveness of the perception system on system performance.
Examine how the disturbance estimation approach taken performs vis-à-vis an approach employing a
higher fidelity slip model. Given its importance, other ground modeling issues could also be
systematically addressed.
Autonomous Mobile Information Collection Using a Value of Information-Enriched Belief

Approach
Injection of human knowledge can significantly increase the mathematical tractability of motion
planning problems. As such, this research could investigate how this can systematically be done as
mission is progressing (constraints permitting).
93

Intelligent Mobility (Minitar and RoboSimian)
It appears that the hardware is supported by useful simulation models and lower order control-
relevant models. This could be carefully quantified by showing hardware data alongside supporting
model-based simulation data. It would be good to see these platforms being more fully exploited by team
members.
Deductive, Analogical, and Associative Reasoning in a Semantic Vector Space
The biggest question on this work is who else is doing related projects. There is certainly related work
that is kept proprietary within Google and other companies; there is almost certainly relevant work within
the federal government. It will be important to stay in touch, as much as possible, with relevant work in
those and other communities. This research could more fully utilize relevant semantic vector space
literature in order to achieve the desired “Watson-like” reasoning. The research could examine what types
of questions are mathematically tractable (polynomial time) and which are not (nonpolynomial time). The
insertion of real-time expert knowledge can potentially (and significantly) help with the latter.
Context-Driven Visual Search in Complex Environments
This project will develop “focus of attention” based on mission constraints, temporal constraints,
semantic constraints, and 3D depth cues. It has direct applicability for robot sensor pointing and processor
allocation in performing real robotics missions.
It is straightforward to see how individual constraints will get integrated. It is more interesting to see
how constraints will evolve over time: once the perception system detects a window, for example, it is
likely that other windows will be nearby and aligned; how will that sort of information be incorporated
into the sensor aiming strategy?
The use of a pan-tilt-zoom camera for context driven search is very important. The researcher could
show how the pan-tilt-zoom camera reinforcement learning approach can be used to incorporate battle-
relevant temporal and semantic mission constraints.
Many other extensions to the research are possible, such as multiple hints coming in from the human,
perhaps asynchronously during mission execution requiring real-time replanning; multiple robots doing
the exploration; mixtures of air and ground vehicles; and surveillance of moving targets.
This research needs to carefully examine time-accuracy-geometry-human-intervention trade-offs and
plan a transition to TARDEC and other customers.
Parsimonious Online Learning with Kernels
This is a very basic machine learning and function approximation technique. It appears to do near-
optimal clustering with online learning to create an efficient and sparse data representation. It is not clear
how sensitive the parameters are, or how much tuning needs to be done for different applications. Future
work could demonstrate the advantages of this method in a concrete and practical example. The research
uses a kernel approach for online sequential sparse classification. The research could systematically
compare this approach with other approaches and also examine battle-relevant applications.
94

Optimized Output Codes for Deep Constrained Neural Nets
The work proposes a new encoding of neural network outputs that shows superior performance in
terms of similar outputs to similar inputs and in terms of resistance to perturbation. The examples shown
are of a relatively small case, learning recognition of hand-written digits after having been trained on
hand-written numbers. It would be important to show similar advantages on larger and more diverse data
sets. This research examined how output codes can be optimized to assist with classification and lifelong
learning issues—for example, reduction of catastrophic forgetting. It examined logistic regressive pairs,
spectral hashing, and latent codes (zero shot learning). Two data sets were used. This research could show
how this optimized code approach can be used within a realistic intelligence and control group battle-
relevant benchmark.
MACHINE-HUMAN INTERACTION
The Army is positioning itself for a future in which humans and machines will closely collaborate to
accomplish mission goals in dynamic, unpredictable environments. To this end, the machine-human
interaction (MHI) effort focuses on developing basic research that will allow a machine to effectively and
safely team with humans. The specific challenges being addressed include creating models of shared
cognition in order to improve team performance, using multimodal sensors and technologies to promote
effective communication, predicating a robot’s behavior on social and cultural information, and verifying
the safety of new, artificial intelligence-driven technology in simulation prior to deployment.
ARL makes a distinction between machine-human interaction and human-machine interaction (HMI).
MHI focuses on the development of algorithms that will allow a machine to more effectively
communicate and act as a teammate to humans. HMI, on the other hand, examines the ways that people
interact with robots. These closely related subfields of human-robot interaction (HRI) use different
perspectives to look at the central HRI problem: How do and should people and robots interact? The work
understandably combines simulation and research on real robots. Simulation experiments offer a means
for rapid prototyping and inexpensive experimental validation. Research on real robots, although more
challenging and expensive, is critical for verification of preliminary simulation experiments.
Transparency with respect to machine or robot behavior is an important underlying goal of ARL’s
MHI campaign. Projects within the campaign attempt to produce not only accurate machine behavior, but
behavior that will be understandable to a human operator and result in greater team performance, trust,
and lower human workload.
ARL has a variety of MHI projects crosscutting a number of different human-robot interaction (HRI)
problems. The projects that the panel examined are each at different stages of development and maturity.
As a whole, the projects demonstrate both opportunities and challenges for the efforts in this area.
Wingman Software Integration Laboratory
The Wingman Software Integration Laboratory project integrates autonomous control and targeting of
an unmanned high-mobility multipurpose wheeled vehicle weapon system with a manned vehicle,
resulting in a human-machine combat team. The project’s initial focus has been on the creation of an
integrated simulation and testing environment that utilizes data from test locations in Michigan to
generate realistic simulation and training situations for soldiers. The simulation environment successfully
integrates the Unity game engine for targeting and the Anvil game engine for autonomous vehicle control
95

using standard methods for communicating between game engines. This integrated system acts as a
realistic software-in-the-loop simulation test-bed for the purpose of rapid prototyping in real-world
vehicles and realistic experimentation of human-machine decision making and behavior.
Although only 6 months into the project, it is evident that this project is progressing rapidly and has
already generated a number of significant accomplishments. The project has produced and demonstrated
an initial version of the simulation environment for testing. These accomplishments are partially due to
the partnerships between the research team and teams responsible for fielding autonomous components.
Potential operational environments and concepts as discussed and developed at weekly meetings with a
number of Department of Defense (DOD) partners including the U.S. Army TARDEC, the U.S. Naval
Surface Warfare Center Dahlgren Division, the U.S. Army Armament Research Development and
Engineering Center (ARDEC), the U.S. Army Maneuver Center of Excellence, the U.S. Army Test and
Evaluation Command, and DCS Corporation. This large number of collaborators is evidence of
significant Army support for the project. The project has resulted in the creation of several technical
reports; while there are no publications to date, it is not surprising, given the short period of this project.
Data collection that is planned includes a simulation event at ARL to assess a warfighter machine
interface for the roles in January-February 2018 and use of the Wingman System Integration Laboratory
for training and human subjects’ data collection during warfighter experimentation in June-August 2018.
The results from these experiments will be critical for determining the long-term applicability and impact
of this research.
Leveraging Mutual Information to Enable Human in the Loop
This project explores the development of a type of sensor fusion application that combines
physiology-based human neural classifiers generated from electroencephalogram (EEG) data with
computer vision-based classifiers for object detection to create classifier ensembles that accurately
identify objects in an image. The project examines a rapid serial visual presentation task in which a
human subject is briefly presented with an image and must identify target items in the image. A
significant portion of this research uses mutual information to evaluate the relevance and redundancy of
the generated classifiers in relation to the target. Ideally, a set of highly relevant, minimally redundant
classifiers will be identified maximizing performance. Results to date have been a mix of theoretical and
empirical studies. These results indicate the strong potential of this approach to identify relevant
classifiers but are less able to identify redundant classifiers.
This project examines natural bidirectional dialogue for human-machine teaming and collaboration.
The project focuses on a search-and-navigation task in which a human commander uses dialogue to direct
a robot during a search task. To date, the project has focused on the collection of large corpus of data that
will be used to automate portions of the overall system. One problem identified is that a large percentage
of the data has not been collected from realistic users of the system, such as soldiers and army operators.
The directions generated by real users may differ significantly from the directions created by university
students or other naïve populations.
Assessing Vulnerabilities in Autonomy
This project attempts to assess the vulnerability of an autonomous vehicle convoy to attack by
opposing forces. Although the project is well motivated and important, it lacks a well-defined research
problem, realistic scenario, articulated metrics of success, and scientific approach. The project’s primary
96

results to date are the recognition that a trade-off exists between system safety and vulnerability and the
recognition that subsystems of the overall system will also play a role in overall system vulnerability.
ARL has a unique opportunity to conduct soldier-centric MHI studies. Conducting soldier-based
studies would ensure both that the research conducted is impactful for the end user and that the work is
grounded with respect to real-world applications. It has been noted, however, that there are significant
challenges associated with the use of soldiers as human subjects. Among these are the fact that soldiers
are an Internal Review Board protected population, that soldiers are no longer located at the ARL
Aberdeen location, and that soldiers may not have incentive to participate in such studies. On the other
hand, it is also noted that ARL often hosts West Point cadets who could also serve as an important human
subject population. Overall, ARL is better positioned than other research centers to perform soldier-
centered human subject studies, and the verification of these technologies could be accomplished using
soldiers. It will be important to include soldiers, to the extent possible, from the beginning of the project,
in order to ensure that the work proceeds in a direction of value for its end customer.
It is also critical that ARL MHI research focuses on realistic army missions as study scenarios. While
these missions need not be so realistic as to warrant being classified, notional realism is nevertheless
critical for ensuring the development of technologies of value to the Army and the DOD.
Given ARL’s expertise and resources, an opportunity exists to investigate and solve problems that, in
some cases, are more complex than the problems tackled by academic researchers. Academic researchers
may be drawn to collaborate with ARL on these realistic problems. One challenge to developing and
maximizing the potential of these collaborations is the ability of ARL researchers to share code and data
with extramural partners. ARL has had some success creating an Open Campus Initiative allowing
extramural researchers to more easily work with ARL researchers. But the success of Open Campus
Initiative is uneven, with, for example, easier access for researchers at Adelphi than at Aberdeen. Further,
not having an open network makes code and data sharing difficult.
Also, it is important that projects have a metric of success from the outset. These metrics will help
researchers remain focused on the project’s research question, even in the case of Blue Sky projects.
Leveraging Mutual Information to Enable Humans in the Loop
This work would strongly benefit by using, to the extent possible, soldiers and intelligence analysts as
human subjects. Subject matter experts (SMEs) and realistic operators may use unique heuristics resulting
in significantly different performance than naïve subjects. This would likely impact the types of
classifiers identified as relevant. Moreover, using images from realistic scenarios could also influence
performance and the types of classifiers identified as relevant and redundant.
The project employs a “Wizard of Oz”-style setup to create dialogue data and intends to move away
from real environments toward simulation environments in order to speed up the data collection process.
While the movement from simulation is understandable, it was noted that the experimenters will need to
keep the subjects convinced that they are controlling a real robot. Providing computer-generated images
of a simulated robot and environment may alert the subjects to fact that they are not really controlling a
robot.
97

Assessing Vulnerabilities in Autonomy
This project needs a well-defined and scoped research question with significant customer
commitment. One possible approach might be to look at case studies of convoy attacks and identify
common sets of vulnerabilities from these attacks. The project also needs precise metrics and measures of
success. It is not currently clear if the project is attempting to generate as many vulnerabilities as possible,
consider unique vulnerabilities resulting from autonomy, or some mix of these two. Last, the project
needs to increase communication and buy-in from the customer. Weekly conference calls focused on
specific, incremental project goals, rather vulnerability brainstorming, might keep the project focused.
PERCEPTION
ARL seeks to be a premier research organization whose discoveries and innovations successfully
transition to the field and support the Army’s long-term strategy of land power dominance. In pursuit of
this vision, the perception group has committed to a long-term program emphasizing the key campaign
initiative—Force Projection and Augmentation through Intelligent Vehicles.
The perception group’s activities have focused around their fiscal year (FY) 2020 goal: “Semantic
labeling of an increasingly larger vocabulary of objects and behaviors to permit a richer, more detailed
description of the environment.” Additional activities emphasize the practical aspects associated with
ensuring correct spatial interpretation of sensory signals, so that the environmental descriptions are
spatially accurate. For this review, evaluation of the group’s output is assessed according to the stated FY
2020 goal, to its potential impact to the field of perception, and to its long-term link to the key campaign
initiative, as well as the current campaign—Sciences for Maneuver.
The perception group’s research activities are tightly focused on advancing theoretical and practical
aspects of learning and estimation tasks of importance to Army capabilities and domains. Areas of study
include object detection and learning, action learning, environmental learning, perception on robot
platforms, and online parameter estimation for calibration of robot sensor suites. Importantly, all of the
projects support progress toward the stated FY 2020 goal.
Overall, the presentations and demonstrated projects represent solid advances in their fields. The
majority of the work indicates awareness of current research activities in the field of computer vision. The
group’s research has stayed at the cutting edge, embracing new successful methodologies in computer
vision and advancing existing approaches through intelligent analytical insights into the problem at hand.
The group has successfully leveraged past external collaborations to strengthen its intellectual
capacity, consequently strengthening these ties. The group demonstrates a commitment to maintaining
high visibility of the work through dissemination of research in conferences and, to a lesser degree,
journals.
The perception research can be broadly categorized into several key topical areas, which are
summarized and discussed in the following paragraphs.
Object Learning
The recently initiated APPLE project focuses on techniques for object learning and refinement using
fused color and depth data. To date, this project has generated useful reports on the state of the art, good
progress on technology selection, and a roadmap for data collection. In the future, this project will
integrate multiple technologies for shape and appearance capture, model creation and refinement, and
98

representation. It is necessary to carefully choose data and modeling domains that are maximally relevant
to Army needs and scenarios. A related (and also new) project in object learning seeks to assess the
impact of embodiment on agent-assisted training of humans to provide useful object views to an image-
based model learner. As these projects mature, there will be an opportunity for cross-pollination of ideas.
Object Detection
One project and demonstration addressed the task of object detection—specifically, techniques for
improvement of deep learning-based detection. A novel region-of-interest proposal mechanism provides
“side information” that can be used to augment the set of examples used during training. Use of this
additional information demonstrates some improvement in performance on an academic computer vision
benchmark, and the demonstration included a close-to-real-time performance level. This work is
interesting, but demonstration and performance characterization on data sets with close relevance to Army
operations would be useful to assess the potential impact of the work.
Action Learning
Two projects address learning and representation of actions. One of these projects contributes a
notable advance over the state of the art—namely, an unsupervised method for learning action attributes
from data and segmenting video sequences into action primitives, which serve as a compact signature for
the activity. A topically related project integrates textual features during training to improve the
performance of a deep learning-based activity recognizer. There is an opportunity for these two projects
to enhance one another.
Environmental Learning
ARL staff demonstrated an impressive integrated sensor suite for environmental sensing. This
platform includes multiple depth and red, green, blue (RGB) sensors, easily and jointly calibrated using a
fiducial. Data from this suite and lessons learned from integration could inform sensor platform
configurations for future mapping robot designs. Also in this category is a project that integrates terrain
and map data with hyperspectral measurements to perform improved classification of water and nonwater
regions. Although narrowly scoped at present, affordable near-infrared (near-IR) hyperspectral imaging
may add a useful new tool to sensing suites used in support of land operations.
Online Calibration of Proprioceptive and Exteroceptive Sensors
Known positioning of onboard sensors is essential to the correct spatial interpretation of data for
modeling and estimation purposes. Having correctly calibrated sensors guarantees proper description of
detected object and recognized activities within a world frame. ARL staff presented two projects covering
calibration of onboard sensors, proprioceptive calibration using recursive filtering and exteroceptive
calibration using a graph optimization. In both cases, the approaches are grounded on strong theoretical
principles and numerical methods. Furthermore, the research activities are focused on problems that have
contemporary value and are still insufficiently investigated by the perception community. The work is
mature and demonstrates strong potential to transition to use within ARL’s robotic platforms, as well as to
lead to new algorithms for maintaining correct sensor calibration during operations.
99

Perception on Robot Platforms
Several robotic demonstrations incorporating perception elements were provided and were generally
impressive examples of technology integration. In addition to the multisensor platform and the object
detection demonstrations mentioned earlier, the RoMan, Minitar, and RoboSimian platforms will offer
highly useful test-beds for joint sensing and manipulation task embedding.
Perception research has gone from being model based to data driven on the strength of statistical
machine learning and deep learning algorithms. At the moment, industry needs to dominate the target
application domains of researchers and the data being collected. In the case of deep learning, ever more
massive data sets are created by the collective efforts of the research community, sometimes with funding
from industry. This situation is both a challenge and an opportunity. It is a challenge because Army needs
are not necessarily serviced by these data sets. It is an opportunity because the generation of an appealing
and challenging data set with long-term Army significance could easily be picked up by the research
community through proper targeting. In doing so, ARL might even benefit from community contributions
to the data set, as well as from the creativity of the research community with regard to ARL-relevant
applications.
In a related vein, deep learning architectures are diverse and their engineering has supplanted the
traditional feature engineering strategy of the previous decade. ARL’s ability to attract new,
knowledgeable talent will determine whether it keeps current with the trends in perception. Due to the
rising popularity of machine learning, and deep learning in particular, ARL is in a good position to hire
more experts in this area by exploiting existing academic relationships and cultivating new ones. The
perception group has the opportunity to take advantage of the pipeline of experts graduating in machine
learning, for both robotics and perception. Achieving critical mass in this area would help promote their
stated goals.
The mid-term (FY 2026) goal embodies difficult challenges that need significant research effort,
including inference, dealing with context, and extracting relationships between objects. The goal is stated
as follows: “Creation of the ability to infer purpose from the relationships between objects in the
environment and behaviors (activity) exhibited by people (teammates, adversaries, and noncombatants)
and place objects and behaviors into context.” The perception research community has not yet fully
embraced activities that would support this end-goal. It is not clear to what degree model-based methods
and data-driven methods will be needed, or combined, in achieving it. ARL has the opportunity to define
canonical problems in this arena, as well as to curate unclassified data sets and scenarios that could both
help push the state-of-the-art in this area of perception and be of utility to ARL mission scenarios.
In general, research presentations and posters were professional, logical, content-rich, and useful.
Clear growth in knowledge content by ARL researchers and support staff was demonstrated. Significant
advances in the use of analytical and simulation tools were observed. The collaborative interactions—for
example, the Collaborative Technology Alliances (CTAs) and Collaborative Research Alliances
(CRAs)—continue to be productive. The Panel noted the various director-level responses to previous
Panel recommendations. These positive responses are also reflected in the continuous improvement in
Campaign research performance.
100

of information-enriched belief approach (projected functional stochastic gradient-based approach with

teams of robots); and research and simulation work on Wingman Software Integration Laboratory, which
has a clear path to Army-relevant static and dynamic scenarios and multiple-machine and multiple-human
interactions.
The overall technical quality of the intelligence and control effort is good, and has shown continual
improvement—particularly since the 2015-2016 assessment by the ARLTAB. The group has benefited
from the hiring of highly skilled postdoctoral researchers, some of whom are being groomed to become
full-time ARL employees. Publication in peer-reviewed journals and participation at professional
conferences has continued to grow, coupled with increasing participation in other professional activities.
Collaborations with peer communities and reputable academic groups appear to be healthy, and provide
the researchers with invaluable networking opportunities and options to leverage quality research
elsewhere. The investment in quality R&D, especially in areas less likely to be pursued by academia, has
increased the potential for impact. The connections between the individual research projects and the
Collaborative Technology Alliance (CTA) and Collaborative Research Alliance (CRA) programs are very
useful and are highly commended. While it would be a mistake to expect all basic research to be tied into
the CTAs, the CRAs provide rich sources of data and research problems, and ready platforms for
integration and testing in a research-friendly environment. The CTAs and CRAs may naturally serve as a
starting point for the benchmarks mentioned earlier.
The research generated by the MHI group is generally of high quality, and is focused on important
MHI areas and largely comparable to university-led research. In particular, the posters and presentations
typically contained acceptable technical content, experimental methods, presentation of data, and
statistical analysis of results. The research reflected a broad understanding of the science and references to
related work, indicating knowledge of research conducted elsewhere. The qualifications of the research
teams were well matched to the research problems and employed acceptable and often state-of-the-art
equipment and models. The research typically utilized an appropriate mix of theory and experimentation
to arrive at well-reasoned conclusions. The Wingman Software Integration Laboratory was identified as a
promising project potentially resulting in outstanding data and knowledge that could ultimately be
transitioned to the field. The project is focused on an important topic, necessary for the deployment and
implementation of human-machine teams with automated targeting. ARL has a strong set of well-
qualified MHI researchers addressing important, Army-related problems. These researchers have a unique
opportunity to generate mission-critical data from a population of specifically trained human subjects.
Doing so would increase the impact and applicability of the research while also helping the researchers
better understand the needs of the population they serve.
Overall, perception research is addressing cutting-edge problems, with meaningful and relevant
results. The group demonstrated an appropriate mix of theory, computation, and experimentation. The
group’s publication list and strategy spans the gamut from respected, application-based conference venues
to well-regarded academic conferences and publications. There is an opportunity for the group to extend
and enhance key projects to yield publications in the field’s very best journals with some regularity.
When considering the collective portfolio of researchers at individual, leading universities or laboratories,
the work achieved by ARL is comparable in scope and outcome. Together, the perception group’s
projects reflect an understanding of relevant state of the art, while demonstrating a commitment to
pursuing key open questions of Army relevance. It is clear that ARL has attracted well-qualified research
staff and provided them with excellent facilities for conducting cutting-edge research in perception.
Several of the projects were particularly well presented and showed strong promise to transition to Army
use. One such project is the online gyro calibration algorithm, while another is the embodied training
project. They demonstrated solid understanding of the tactical ARL end point while bringing together the
proper theory or practice, as needed.
101

Several opportunities are identified for even greater advancement in the Campaign research
productivity. These include need to increase level of effort in several 6.1 and 6.2 research projects and
internal collaborations; need to increase mentoring of junior research staff; need to increase use of Army
(soldier) field experiences and scenarios, robots, and more relevant data sets in all Campaign research;
need to address systematically the complexity, scalability, robustness, uncertainty, and operations in noise
and interference, that is, boundary operations; need to establish metrics and benchmarks within a push-
pull research context; need to increase ARL presence and participation in journal publications and
conference papers so as to define the problem set; and need to increase strategic, collaborative
engagements with industry including via CTAs and CRAs.
According to the U.S. Army Operating Concept,1 human-machine interaction and teaming will be an
important near and intermediate focus of research for the army. The projects that were reviewed appear
well positioned to provide mission-critical data and technologies toward developing enhanced human-
machine teams. For the most part, these projects are progressing well and generating comparable
technical quality to academic research. This finding is supported by the fact that the four projects
reviewed have collectively published eight papers at peer-reviewed conferences or workshops. The
venues for these publications are the same venues as academic researchers.
To date, the MHI research reviewed underutilizes military personnel as human subjects. This finding
is supported by the fact that the only ARLTAB reviewed project that has used military personnel is the
Toward Natural Dialogue with Robots: BOT Language project and this project will continue to recruit a
mix of military personnel and civilians as human subjects in the future, as available. MHI project
researchers could, to the extent possible, use soldiers, cadets, and realistic army operators.
Currently, the MHI research reviewed underutilized realistic mission scenarios with military
relevance. This finding is supported by the fact that three of the four projects reviewed did not employ a
realistic mission scenario and relied on somewhat contrived notional missions. MHI project researchers
could use realistic relevant mission scenarios whenever possible.
ARL is conducting high-quality perception research that addresses important issues toward the near-
term (FY 2020) goal of “semantic labeling of an increasingly larger vocabulary of objects and behaviors
to permit a richer, more detailed description of the environment.” ARL has built up a strong group of
researchers in perception, with appropriate resources and facilities to conduct important basic research.
ARL has done a good job of disseminating their research through top-notch conferences, and has formed
strong collaborations with external partners.
Given that much of the ARL perception research is currently being evaluated on commercial or public
domain data sets with limited obvious relevance to Army missions, an open question is whether the ARL
research will perform similarly when it is ultimately transitioned to Army applications. It is well known
that the performance of machine learning algorithms varies when different data sets are used. ARL may
discover that its research is not addressing the right problem or the right solution for Army-specific
applications. It is thus critically important for ARL perception research to be validated using Army-
relevant data. While recognizing the challenges of generating such data, performance evaluation of the
developed approaches to perception for Army-relevant missions is not possible without doing so, and
without validating again said data.
Recommendation: ARL should develop a research emphasis on the generation of Army-

relevant data suitable for advancing data-driven perception methods and evaluating research in
perception against mission-relevant outcomes.
1
The U.S. Army Operating Concept (AOC): Win in a Complex World—2020-2040, TRADOC Pamphlet 525-3-
1, U.S. Army Training and Doctrine Command, Fort Eustis, Va., October 31, 2014, http://www.tradoc.army.mil
/tpubs/pams/tp525-3-1.pdf.
102

Many, but not all, of the active research projects are clearly motivated and informed by an Army-
relevant use case. While motivation arising from general improvements in the state of the art and
problems motivated by academic projects can be useful, a closer linkage to Army priorities and the
technical roadmaps articulated by the group will help bind research and researchers more closely to
organizational goals. Further, the research can be conducted with the ultimate sensor or platform
constraints in mind, keeping in mind that vision-based research could eventually be embedded on mobile
platforms in the field. Such constraints might influence the choice of approaches pursued in perception
research. Such a linkage will help inform the project regarding platform constraints for deployment.
Recommendation: ARL should closely match all current projects and all new starts to a service-
relevant goal in the organizational roadmap and employ platform/deployment constraints as
research planning parameters. ARL should consult existing robotics roadmaps and
organizational priorities, and develop a story line showing how existing and new efforts feed
together to develop the desired future capabilities.
While the ARL vehicle intelligence research is of high quality, most of this research is at the single-
investigator level. Even projects that have closely related objectives or approaches are conducted without
strong connections between them. Furthermore, it is not clear to what degree existing functional robot
demonstrations capitalize on prior ARL research. Identifying synergistic ties between related projects
(such as the use of common data sets, platforms, frameworks, and benchmarks) can speed innovative
progress and increase potential impact. This could be achieved by identifying important Army-relevant
use cases that inform the research projects, and then comparing and contrasting related ARL research in
that context. The research would still be fundamental and basic, but informed by the specific Army
application, as well as the advances made in other relevant areas of ARL research.
Recommendation: ARL should explore incentives and goals for increased internal collaboration
across closely related projects, forming tighter connections among algorithms, sensor suites,
robot platforms, and Army-relevant use cases. ARL reports and studies should describe how
the work fits into the overall ARL mission (with specific customers targeted); what major test-
beds will be exploited; how younger scientists are working with more senior scientists; and what
expertise will be developed in house versus what will be imported from industry, academia, and
other laboratories.
In addition, ARL could accelerate progress in related research areas by developing a strategy for
centralizing internal expertise on nontrivial tools and techniques, such as methods for deep learning.
Centralized expertise would enable projects and experts to synergistically benefit each other, and shorten
the learning curve for investigators who are using techniques closely related to other projects.
Recommendation: ARL should centralize internal expertise on nontrivial tools and techniques,
to shorten the learning curve and accelerate progress on related projects in perception.
While the ARL research in perception is being disseminated in conference publications, the ARL
record is more limited for journal publications. In general, however, the quality of the ARL work is
sufficient for publication in top-quality journals. Such publications would increase the visibility of this
work, as well as the potential impact of ARL research on the state of the field.
Recommendation: ARL should increase the dissemination of its perception research through
top-quality journal publications.
While some benchmarks can emanate from the academic or scientific community, some need to
definitely come from the military research community. The team could use test-beds and benchmarks to
103

serve as unifying umbrellas for the Sciences for Maneuver Campaign. A 5-year benchmark mission
(possibly virtual or involving, for example, Building 570 with multiple ground and air robots) would help
focus ARL’s efforts and the efforts of individual researchers. The benchmarking effort is expected to
enhance essential and foundational modeling, simulation, and theoretical components.
Recommendation: ARL should identify several benchmarks that can be used to assess methods
being presented and facilitate comparisons of ARL efforts as well as other state-of-the-art
methods. These benchmarks should be ordered in terms of complexity of addressed scenarios
(e.g., images and video clips) in order to systematically assess definitive and appreciable
progress over time and from benchmark to benchmark. ARL should use these benchmarks to
facilitate and measure the associated incremental or successive progress.
104

Human Sciences
The Panel on Human Factors Sciences at the Army Research Laboratory (ARL) conducted its review
of selected research and development (R&D) projects of the ARL Human Sciences Campaign at the
Aberdeen Proving Ground, Maryland, on June 27-29, 2017. The human sciences project areas reviewed
were as follows:
 Real-world behavior. The objectives of the R&D in this area are to enable the collection,
analysis, and interpretation of human behavioral data within dynamic, complex, natural
environments. ARL conducts R&D in the following two areas: (1) real-world complexity
in human science experimentation, and (2) assessing human behavior in the real world. A
key focus of this work is the development of novel technology and methodologies and to
collect and analyze these data in real-world conditions.
 Human variability. The goals of this R&D are to enable high-resolution, moment-to-
moment predictions of an individual soldier’s internal and external behavior and
performance and the ways in which soldiers interact dynamically in mixed-agent team
and social settings in both training and operational environments. Human variability
R&D is conducted in the following two areas: (1) multifaceted soldier characterization to
develop a comprehensive understanding of the factors influencing human variability, and
(2) brain structure-function coupling to create a multiscale understanding of the
relationship between the brain’s physical structure, its dynamic neurophysiological
functioning, and human behavior.
 Humans in multiagent systems. These efforts aim to achieve critical technological
breakthroughs needed for future Army multiagent, mixed-agent teams to effectively
merge human and agent capabilities for collaborative decision making and enhanced
team performance in dynamic, complex environments. The challenges for human
sciences R&D in this area are soldier workload, situation awareness, trust, influence, and
cultural cognition.
 Human cyber performance. This program aims to advance a foundational science of
cybersecurity that addresses the human dynamics of attacker, defender, and user
interactions in Army networks to support training effectiveness and transition of agent-
based technology to improve the operational efficiency and effectiveness of cyber-
warfighters.
REAL-WORLD BEHAVIOR
The Real-World Behavior (RWB) Initiative is an ambitious program that extends research from the
laboratory to the field with the aim of understanding and predicting human behavior in naturalistic
environments. It examines potentially unpredictable operational and stimulus environments, open
behavioral responses, and potentially highly sensor-rich and pervasive monitoring. For this reason, one
105

important focus is to develop novel technology and methods for sensed measures from neural and
physiological data in the individual and the interactive behavior of individuals and groups in teams.
The progress of this group, including the choice of initial projects, is impressive. They have
developed state-of-the-art enabling technology in the areas of electroencephalogram (EEG) and vehicle
instrumentation, and other projects investigate potential approaches to analysis of complex data sets. The
use of technical and quantitative methods and approaches to experimental investigations are at a high
standard; however, important theoretical discoveries and approaches remain in the early stage of
development. The portfolio has a good balance between work in the laboratory and in the field,
sophisticated analytic approaches, and the preliminary development of theory. The group has significantly
advanced its capabilities and agenda since the 2015 review,1 while demonstrating strong productivity and
leadership roles in collaborative work and field consortia.
A goal of the RWB initiative is to approach this new science as an iterative process that first takes
theories and findings from the existing literature and laboratory experimentation and embeds them into
carefully instrumented and designed tests in real-world contexts. After harvesting new discoveries and
theories from the necessarily less controlled—but more ecologically valid—real-world research, the
second step would test these new ideas in carefully controlled detail in a new generation of experimental
studies.
The existing projects, some reviewed here, pursue enabling technology, test models for analysis of
RWB data sets, and test real-world inspired challenges in the laboratory setting. The group has made
significant progress on enabling technology since the 2015 review. It is ready to collect data in several
projects designed for the field with complex instrumentation for multisensor data. Several experimental
laboratory projects are either inspired by the gaps between standard experimental testing environments
and the real environment, or are designed to pretest and validate methods of analysis of data from ongoing
neural and other sensor data in open behavior scenarios.
One of the unique features, shared among several initiatives, is the focus on developing technologies
to enable real-world deployment of research. Several of the initiatives take advantage of the strength in
materials sciences, chemistry, or computing sciences within the ARL—a combination of expertise that
can be found in some academic centers but would be difficult to duplicate and coordinate within research
environments in most industrial settings.
Enabling technologies have been developed for EEG, a neural signal useful for indexing human states
with potentially feasible field deployment. The group now has state-of-the-art expertise in EEG systems
and source localization; it has developed in-house EEG technology and compared it with commercial
EEG systems. Of particular note is a head phantom for EEG, a device approximating the human skull
conduction used to recreate electrical signals on the scalp that will enable the modeling and exclusion of
noise sources, with the goal of identifying measurable neural signals recorded in complex environments.
The group has developed a cutting-edge facility for integrating EEG and other related neural and
physiological sensor data.
The target for future work is to further enable technology. The number-one problem of recording
EEG in the field will be powering the measurement and analysis equipment indefinitely. An important
project created a novel integrated approach to amplification and digitization resolution in an online,
adaptive manner that will enable low-power operation for applications such as EEG that are capable of
being powered from locally harvested sources (e.g., thermocouple or solar power). This type of device is
unique to the real-world recording environment and is currently not available commercially.
1
Research Laboratory: Interim Report, The National Academies Press, Washington, D.C.
106

RWB data collected over extended time periods during complex human or human-agent behavior
makes critical demands on data management and sophisticated data interpretation. The data may include
multiple sensor readings, patterns of communications, behaviors, and behavioral outcomes. Important
efforts are being made to create an approach for analysis and interpretation.
A data storage and integration system centered on EEG has been developed, including methods for
synchronizing multimodal measurements and a standardized format for data. The system is extensible to
pulse, galvanic skin response, respiration, accelerometer, position, velocity, vehicle state, text, natural
language, and in some cases eye tracking (ET). It provides an approach to the aggregation and cross-
referencing of big synchronized data streams. ARL staff have participated in and in some cases led the
development of these standards through the Big EEG consortium.
Measuring neural responses during natural eye movements is one target of study. Real-world
recording of EEGs contains several additional requirements above and beyond the typical laboratory
environment. Until quite recently, saccades and fixation-related potentials have been either ignored or
constrained using fixation points. Cost and complexity of eye-tracking (ET) equipment has been the
primary limitation of merging these technologies. ET equipment has improved dramatically in the last 10
years. The number of manufacturers of ET equipment has increased, providing more accurate, faster, and
portable equipment. Many ET equipment manufacturers now provide interfaces to a variety of EEG
equipment. Likewise, EEG equipment manufacturers are starting to provide support for a variety of ET
systems. However, these systems are aimed primarily at the larger marketing and product development
community and are not ideal appropriate for the RWB goals of this team. Further technology for
ambulatory ET and EEG remain to be developed.
Related laboratory projects examine eye-fixation synchronized EEG as a potential approach to
understanding EEG with free viewing. Extrapolating from the classic but simplified gaze-constrained
experimental measurements, neural signatures of target detection, attention, and human state are indexed
to the onset of an eye fixation during task performance. These studies are beginning to yield important
basic findings about the neural signatures of goal-directed behavior. Fixation-related neural potential is an
emerging area of research, with very few published articles in the area, and this research can form the
baseline study for further analysis. The research study relating the neural response to the saccadic distance
traversed by the eye and the spatial frequency of the stimulus is a unique and valuable contribution to this
nascent field. To our knowledge there are no previous studies on these critical components for real-world
recordings.
Another pair of projects motivated by the gap between typical laboratory measurements and the
typical conditions in the field study performance under high dynamic range (HDR) luminance conditions.
These two projects, together, are an example of how this group is taking methods and results that have
been studied in the laboratory, and testing how these methods and results generalize to much more
realistic contexts. Human and machine vision have mostly been studied with stimuli and images that have
limited dynamic range (computer display screens). However, human and machine vision systems could
deal with much higher dynamic range (e.g., brightly lit scenes) in the real world. Visual target detection
and visual search have been thoroughly characterized for low dynamic range stimuli using well-
established visual psychophysics experimental protocols and computational neuroscience theories. The
team is in the early stages of extending these empirical and theoretical methods to HDR stimuli. Only
recently has this been made possible, because of the availability of HDR displays. The approach is
technically strong, the topic is important, and the research is likely to have scientific impact.
Another cluster of projects focuses on shooting performance, a domain with obvious applications.
The study of shooting performance as a function of weapon-ammunition configuration is an outstanding
example of the group’s applied research. The objective was to measure performance (accuracy)
degradation for single shot, controlled pair, and automatic burst sequences, across weapon systems with
different recoil energy. The experimental design and implementation and the data analysis methods were
technically sound. The results were clear and readily interpretable, offering a new metric for evaluating
trade-offs between small arms systems. The project studying the role of human fatigue in the speed,
accuracy, and commission of friendly fire commissions tested marksmanship while distinguishing
107

between friendly and nonfriendly targets. This project showed that, while the accuracy of the firing was
little affected, mental fatigue induced by previously completing a complex cognitive task significantly
increased the number of subsequent friendly fire events—failures either to identify a friendly target as
such or to withhold action in this rapid-fire context. This study is a dramatic demonstration of the
importance of human state in these situations. Another field demonstration also showed the strong
potential role of physical sensor systems that can support and guide marksmanship training.
The group has instrumented an automobile used to study authentic brain and behavior during driver-
passenger interactions on a local highway (I-95). The goal of this research is to enable quantitative
measurement of human behavioral performance and brain activity while an individual is driving in traffic
on the highway. A passenger car was instrumented to record synchronized continuously streaming data
about the state of the automobile (e.g., speed, lateral position in lane, and distance behind leading vehicle
in the same lane), and the state of the driver (e.g., pulse, respiration, and EEG). Doing so is a technical
tour de force and an example of the team’s technical capability for measuring human performance in real-
world scenarios. The study is designed to have different types of controlled experimental manipulations
(verbal communication between passenger and driver, amplitude-modulated podcast auditory
stimulation). These are good examples of the team’s creative approach for introducing experimental
manipulations in real-world scenarios. The audio podcast, for example, will enable measuring steady-state
auditory evoked potentials, a rigorous and robust approach for analyzing EEG measurements. The data
acquired with this platform will offer a test-bed for assessment and for further developing the team’s
technical infrastructure for handling big data (database, workflow) and data analysis methods. The
approach is technically strong, the interdisciplinary topics being addressed are important, and the research
is likely to yield results that will have scientific impact.
One study, related to others studying the integration of humans in systems, studied situation
awareness and preservation of communication ties in disrupted environments. It investigates how social
networks (here team networks) measured by communication acts morph as a result of disruptive events.
Collected in the context of a joint (United States-United Kingdom) forces coalition training exercise, the
project observed changes in the patterns of e-mail communications in response to certain identified
disruptive events, based on 20,000 e-mails from 87 individuals in specific roles. Communication ties
reflect the numbers of outgoing and incoming e-mails between individuals when the e-mails may be more
costly to generate than to receive. The study found that the active networks responded differently to
different kinds of disruptive events. This study will be further developed to codify types of disruptive
events and the corresponding predicted changes in communication network dynamics. This information
may inform training activities or may even drive on-the-fly allocation of communication bandwidth in the
field.
Some preliminary studies are focused on understanding the descriptive verbal communication, with
the presumed goal of designing suitable agents for understanding human descriptions and commands.
This area poses many challenges in collection, analysis, and interpretation of complex data. It
provides many opportunities for important and timely developments in both basic research and field
research.
There is a need to enhance the work of this early-state initiative, but if it matures properly, this could
be a program of high potential for a leadership role in the scientific study of real-world behavior in natural
environments, especially if leveraged with other laboratory and extramural resources. The ARL
researchers have opportunities for leveraging research from broad technological partnerships and unique
access to a target population that experiences ranges of human brain states involving fatigue, stress, and
task complexity, both nationally and internationally. The RWB projects also have synergistic interactions
with issues of human variability and the complex demands of multiagent teaming that are also the target
of the Human Sciences Campaign research.
108

The group has done an excellent job of identifying important technological opportunities and
challenges; almost all are high demand and high reward. In most cases, the challenges are also
opportunities, if approached well, to take leadership positions. These include the following:
 To continue work on field-inspired issues such as high-luminance range inputs to vision

and devices, EEG correlates of visual search in complex environments, and others.
 To further enhance enabling technologies for instrumentation in the field for real-time
monitoring, including, for example, size, power consumption, onboard computing, and
transmission and storage of data.
 To address the challenge of interpreting the dense and complex data generated by highly
instrumented continuous collection. Solid field-normed approaches have been identified
based on EEG standards, yet data management, indexing, and interpretation of these data
sets pose challenges.
 To further develop theoretically grounded methods for using real-world events to mark
event-related or reverse correlation analysis of neural or physiological data. A number of
these were already identified—for example, the possible use of intersubject correlation to
identify common states, event analysis by expert annotation, or training markers by
machine learning in the laboratory and extending the classifiers to data acquired in real-
world settings.
One major challenge and opportunity will be the development of technologies and algorithms that
allow real-time data analysis of the relevant neural or physiological measures to support online state
analysis. Another major challenge and opportunity is the identification of efficient or minimal sets, with
appropriate redundancy, in the multimodal neural and physiological collection.
HUMAN VARIABILITY
Many of the efforts within the projects and topic areas aligned with the objectives of two ARL-
defined essential research areas: human agent teaming and accelerated learning for a ready and responsive
force. The human variability portion of the Human Sciences Campaign is divided into two main projects:
Individual Soldier State Dynamics, which aims to develop a more comprehensive understanding of how
the interaction of both trait and state factors account for human variability; and Contextual Influences on
Soldier Performance, which aims to quantify the influence of social and environmental components on
human performance, characterizing how they modulate a soldier’s state dynamics.
Overall, there is a wide variety of research being conducted by a multidisciplinary team of well-
trained researchers.
The number of publications and presentations at scientific meetings has increased over the past
several years. The quality of the journals and scientific meetings at which the material is being presented
is also very good. The efforts that have been made to decrease time for internal review appear to be
working well. The new ARL online operations security and public release process begun in June 2017
will hopefully further streamline approval of publications and presentations to lower the barriers to
publishing nonclassified findings in the scientific literature.
Since the 2015 review, there has been progress in developing equipment and techniques that will
move the research forward. Two such examples are the optical targeting system, which has been
109

developed and used in a study of the impact of fatigue on shooting accuracy, and the EEG technology,
including the soft sensors and the EEG phantom.
Sleep is an important contributor to human variability. Individual differences in sleep duration, sleep
need, and response to sleep loss have been characterized in numerous research studies, and incorporating
such knowledge into ongoing studies of human variability as well as real-world behavior, monitoring
sleep and fatigue of participants in all studies, and integrating sleep into the theoretical framework for
understanding variability in human performance and behavior at ARL is well-justified and overdue.
There are good collaborations with the broader scientific community under ARL’s Open Campus
Initiative. One such example is the connectomics collaboration with the University of Pennsylvania and
the Institute for Collaborative Biotechnologies/University of California, Santa Barbara; such studies
would not be possible at ARL without outside collaborators.
Human variability is a critical area of investigation, because there are differences between individuals
in how they respond to a particular challenge depending on a variety of factors, including (but not limited
to) environmental factors such as temperature, social factors such as role within a unit, behavioral factors
such as relative level of fatigue, and physiological factors such as stress level. While group-based
approaches provide a useful starting point, if systems are designed for average or below-average human
performance, this leaves a lot of untapped human potential.
Adaptive systems that can be trained to assess the individual’s traits and monitor the individual’s
current state and then respond in real time to dynamic changes in state will allow future Army systems to
capitalize on human uniqueness and mitigate against both intra- and interindividual variability. Such
adaptive approaches can provide capabilities to improve physical, cognitive, and social performance; to
decrease time-to-train; and to improve human-network interactions by providing robust predictions of
soldier state and intent to integrate within teams and tactical networks. These adaptive and predictive
technologies will be critical to the individualization of equipment and training, maximizing and sustaining
both soldier and unit peak performance during mission critical tasks.
As noted earlier, the number of well-trained early-career investigators is impressive. However, there
is a notable lack of senior working scientists. To draw an analogy with a typical university structure, there
would be many postdoctoral researchers, lecturers, and deans at a typical university, but few or no
associate and full professors. This suggests that within the group studying human variability, oversight,
coordination, and a view of the “big picture” may be lacking, and the researchers may have limited
opportunities to advance their careers within the ARL structure; if they leave for other opportunities to
advance their careers elsewhere, their talent and experience may be lost to ARL.
There is a need to connect the science of each project to an overall theoretical foundation. There is
supposed to be a balance between field studies, laboratory studies, and the underlying theoretical basis of
each. However, the presentations overall did not reflect the theoretical basis for the individual
experiments. For example, one of the expressed goals was to develop a laboratory setting where high-
resolution, long-term continuous monitoring of individuals can be carried out. While building such a
facility and equipping it are interesting and challenging activities, how such a facility and the resulting
data fit into the overall goals of the human variability program was not well articulated.
There was discussion of monitoring parameters of ARL personnel working in its laboratory setting as
an approach to studying human variability. Related to this large-scale experiment, a plan was described to
do similar monitoring of ARL or other personnel at other sites, including potentially at sites of academic
partners and other Army research facilities. However, how to extrapolate from civilian scientists as test
subjects working 9-to-5 weekday jobs to soldiers making life-or-death decisions in extreme environments
(or if it is even possible) could be considered before, not only after, this activity is undertaken.
Additionally, ARL lacks experience in carrying out multisite trials, and scientists with such expertise need
to be part of any planning and execution of such an ambitious multisite study.
110

In the current experiments and in the experiments planned for the large-scale study, one challenge
will be to identify the type, amount, and duration of data needed to predict the outcomes of interest. It will
be easy to collect a lot of data, but if those data are not useful in predicting outcomes of interest, they will
add burden to the future soldiers without providing any benefit. A plan needs to be implemented for how
key features of data will be selected from the mass amounts of information, with such a plan undergoing
regular updates as new equipment, analyses, and information is developed.
It is commendable that ARL recognizes that individual differences in sleep duration, sleep need, and
response to sleep loss are important contributors to human variability and represent an opportunity to
account for missing sources of individual differences. However, while recognizing that sleep needs to be
accounted for, the current approach of ARL and the current group of ARL scientists do not have the
expertise to move this forward as quickly as it could be done. It is challenging for scientists to enter a new
area of research, and key to doing so in the most effective way is to seek expert advice. However,
selecting the experts to provide advice is challenging when one has insufficient knowledge of the area in
which one is choosing an advisor. While ARL has a current sleep advisor, that researcher’s focus is on
how naps improve memory. This expertise is far too narrow for the broad program of human variability
that ARL is currently working on, and ARL requires a wider range of sleep expertise to advise its human
variability program.
Related to integrating information about sleep into the ongoing studies to understand how it
contributes to variability, biological time-of-day (circadian rhythmicity) needs to be accounted for in all
human studies. In particular, the researchers need to seek to understand how the particular types of
performance or the physiologic measures in their experiments vary with time of day. Human biology is
designed for rest and inactivity at night, yet soldiers need to perform their duties at all times. This has
huge impacts on all aspects of performance, yet it is not considered at all in the current ARL experiments.
The speed of processing information, visual perception, balance, selective attention, short-term memory,
as well as most biological processes, all vary with time of day. Recognizing this is a first step in
understanding how it impacts performance, and that in turn can be used to explore how time-of-day
impacts can be overcome when operational demands require all-day/all-week activities. It is therefore
critically important to recognize that time of day plays a very strong role in both intra- and interindividual
differences in performance, and to incorporate this into ongoing studies. This means that it is not
sufficient to carry out experiments only during the daytime, and it is critical to understand the individual
biological timing of participants in all studies so that time-of-day factors can be accounted for. Outside
collaborators who are expert in studying circadian rhythms in performance and behavior could be
consulted or brought on as partners to ensure this source of variability is considered in experimental
design and accounted for in analyses.
ARL is carrying out numerous studies of human performance, including, for example, shooting
ability, driving, brain function (studied via EEG and magnetic resonance imaging), and decision making.
It would be very useful if there were certain sets of information collected from every participant in every
study, so that common data collected in different studies can be combined for larger-scale analyses, and
so that future data mining activities could be carried out using existing data sets. Baseline questionnaires
consisting of key information could be given to all participants in all studies; individual studies may want
to add additional questions to meet their particular study goals. The baseline questionnaires could be
reviewed from time to time to determine whether additional questions could be added as new information
is learned on factors that contribute significantly to variability.
ARL urgently needs to incorporate cutting-edge biomedical research in genetics, epigenetics, the
various “-omics” fields (e.g., metabolomics, proteomics), microbiome analysis, and biomarkers into all
ongoing experiments. These sciences will very likely provide critically important information about
sources of intra- and inter human variability. The ideal way to pursue this is through a series of
collaborations, whereby experts in the various areas provide advice about samples and data collection,
ARL scientists collect the data, the collaborator analyzes the samples, and together ARL and the
collaborator interpret the findings. It will be critical to select the right collaborators to ensure this effort is
111

carried out effectively and across the various studies. A scientific advisory board could be established to
support this important effort.
Collaborating with academic researchers can extend the range of techniques available to ARL, and
expert collaborators can allow ARL researchers to rapidly expand their work into areas in which they are
not expert. That said, it is critical when moving into new scientific areas that the right collaborators are
selected.
HUMANS IN MULTIAGENT SYSTEMS
The Humans in Multiagent Systems program focuses on problems related to understanding and
supporting distributed soldier collaborations in sociotechnical networks exemplified in three broad areas
of inquiry: human-agent teaming, cyber and networked systems, and understanding sociocultural
influences.
Throughout the description of the research in this area, there were many examples of good
interdisciplinary collaboration to support broad-ranging questions among computer scientists, cognitive
scientists, and human factors psychologists and engineers, such as in the work on trust in robotic
transportation systems. Another example is in the work on mission command in the age of network-
enabled operations, which represents an excellent effort that brings together innovative, multimodal
methods and experimentation in the context of a real application domain for understanding situation
awareness. This is a good example for other potential large-scale research and development (R&D)
studies that integrate empirical, technological, and theoretical advances. The collection of data to support
further investigation into Army-specific communication patterns (e.g., based on rank) shows good
foresight.
The Humans in Multiagent Systems team leverages the foundational (6.1) research of their colleagues
in the Human Variability and Real-World Behavior programs to inform their applied (6.2) work as well as
collaboration with operational warfighters, at Fort Bragg, for example. This serves as a good example of
how cooperation across areas can greatly facilitate progress from basic questions to final products.
The identification of the Cyber Human Integrated Modeling and Experimentation Range Army
(CHIMERA) lab, jointly developed by ARL’s Human Research and Engineering Computational and
Information Sciences Directorate (CISD) for technical cyber research as a target of opportunity for
research into the human aspects of cyber security, represents foresight into outreach and collaboration
with other organizations, and it leverages investments made elsewhere.
The researchers demonstrated a clear understanding of the importance of measures of performance
and measures of effectiveness, that achievement of the former does not always translate into achievement
of the latter, and that this disconnect needs to be addressed in their research. They seek to become leaders
in the development of teaming metrics to more precisely gauge the impact of the tools they are
introducing into human systems and to differentiate when performance improvements are occurring as a
result of teaming changes or other factors (such as the environment or the contributions of exceptional
individuals).
The Humans in Multiagent Systems program is very broad and ambitious. It can become a
mechanism for bringing together a diverse and multidisciplinary group to look at an important set of
Army-relevant problems. On the other hand, the topics of human-agent teaming, cybersecurity teams, and
112

sociocultural influences in civil affairs teams are only very loosely connected, and some areas of expertise
that are lacking. It is not clear whether ARL is gaining significant advantages by the decision to join these
areas within one program.
While the program has done a good job of presenting an overall map connecting identified needs with
research questions, it still remains unclear what the conceptual and theoretical map is that guides
researchers’ specific choice of research projects and questions. For example, in looking at intelligent
agent-assisted route planning, why were mental models and trust selected as variables of interest? Or in
investigating reasoning in civil-military affairs, what is the connection to the existing work on natural
language processing, visualization, and intelligent decision making, and how have the researchers decided
that their particular variables are appropriate? Given the ultimate goal of improving the lives of soldiers,
paying attention to what controls the most variance in outcomes and selecting research questions is
important.
The ARL has an opportunity to become a leader in the area of human-agent teaming. There are some
very strong examples of approaches for evaluating consensus among humans and between humans and
autonomous agents, such as the work on mental models in agent-assisted route planning and the use of
eye tracking in evaluating input from an autonomous vehicle. Autonomous systems researchers expressed
interest in adapting autonomous system reasoning to mission criteria, potentially resulting in more
effective autonomous systems. However, guidance on variable selection and determining where variance
in human-agent teaming really resides are important considerations that seem to be lacking. For example,
there seems to be a large focus on trust, but no sense of why this plays such a large role across studies to
the exclusion of other variables.
In addition, the broader area of artificial intelligence differentiates between technologies for
production and those for coordination; ARL researchers do not seem to be considering this distinction in
thinking about what technologies to explore and the implications of this distinction for this work. ARL
could also try to think even bigger and more creatively about ways that autonomous agents might
fundamentally transform the way the Army and civilians do their work in the future. It is unclear how
much the research work is guided by these bigger, forward-looking ideas versus driven by current
concerns of soldiers and making incremental improvement to current approaches to carrying out missions.
The inclusion of sociocultural influences and civil affairs as a research area in the general ARL
portfolio represents a good opportunity for creating an end-to-end program. Creating tools to support the
gathering, processing, and visualization of data collected by Army civil affairs specialists (and facilitating
stakeholder decision making) appears an important and relevant application domain that can benefit from
new advances in data sciences and machine learning to create scale-up and impact. There is a great
opportunity for creating an end-to-end R&D program that can leverage the multidisciplinary expertise and
approaches while positively impacting the interface between civil affairs and operational commands. This
effort can benefit by laying out some of the high-level open gaps and needs. Finding a way to build on
what is known already about civil affairs challenges to advance knowledge seems to be in a fairly
elementary phase at this point. Similarly, the qualitative research being conducted seems to be fairly
rudimentary compared to standards of similar research conducted in academia, and if this methodology
will be central to the efforts in this area, additional depth may be required.
A significant challenge for all of the research is adapting to the specific demands of operating in the
Army environment. The methods for investigating behaviors and tools similar to those used in civilian
contexts, such as email and electronic chat, need to take into account that the relationships between
individuals vary considerably, constrained by rank, unit, and role. These relationships almost certainly
impact communication patterns and networks to make them behave differently from those reported in
extant literature on social networks. The data collected on network-enabled operations relating Internet
protocol (IP) address to individuals offers the opportunity for significant research into Army-specific
patterns and networks. There is also an opportunity to explore how soldiers work outside the constraints
when those constraints impede critical mission effectiveness. In areas where ARL research is similar to
commercial research, it is critical that the ARL work leverage existing work as much as possible, and that
it then focuses ARL research to address Army-specific situations and needs. For example, robotic
113

transportation and autonomous driving is an area that is robustly funded and researched commercially.
ARL would benefit from focus on areas that are not of commercial interest—autonomous driving over
rugged terrain, for example.
A related challenge is translating technology into the military environment. Soldiers are typically not
early adopters of new technologies; they tend to reject new technologies that do not immediately
demonstrate significant improvement over existing capabilities. Converting autonomous systems from
tools to teammates is challenged by potential soldier expectations that autonomous systems will adapt to
the individual soldier rather than the soldier adapting to the system. It is not clear what assumptions guide
the development of technologies at ARL, and whether thinking about technology adapting to soldiers
might enhance the design of the research to raise the chances that the ultimate products created will be
accepted by the target population.
There is a great opportunity for researchers at ARL to establish themselves as thought leaders and to
create impact by sharing unique data and tools, along with papers. Creating a repository online to make
papers and tools publicly available would facilitate this process.
HUMAN CYBER PERFORMANCE
Over the past year, the small Human Cyber Performance group has established a bold mission:
advance a foundational science of cybersecurity that addresses the human dynamics of attacker, defender,
and user interactions in Army networks to support training effectiveness and transition of agent-based
technology to improve the operational efficiency and effectiveness of cyber-warfighters. Their primary
focus is on the cyber analyst, and they wish to become a leader in the cybersecurity community by
advancing scientific understanding and improving human performance of human cyber analysts in their
critical role in defending real-world Army networks and systems.
To that end they have built the Cyber Human Integrated Modeling and Experimentation Range Army
(CHIMERA) lab. This lab is in effect a human cyber range, and its primary purpose is to enable the
conduct of extensive studies on the cyber analyst in a simulated cyber environment that will allow the
researchers to understand cyber analyst performance in a range of simulated situations. The lab consists of
multiple chambers with displays that are attached to a simulated network environment that allows the
researchers to configure and reconfigure networks of their choosing. Additionally, they are building a
capability to provide a wide range of information to the analysts regarding the real-time status of the
network that would resemble the sorts of output they would receive from standard network monitoring
tools. The goal of the lab is to be able to assess the performance of cyber analysts in a way that has
previously not been available to the cybersecurity community. The lab was under construction and is not
yet operational, so there are no reported results from the lab.
Two recent empirical studies were performed prior to the creation of the CHIMERA lab. The first
was a study designed to understand the extent to which past incidents, reported by cyber analysts, had the
ability to predict future cyber incidents. Using a Bayesian forecasting model, they showed that analyst-
reported incidents from week n had the ability to predict with high confidence the rate of incidents in
week n + 1. The model was not as effective at other time intervals, and it was not intended to predict type
of incidents, only their rate of occurrence. The second study looked at the performance of cyber teams
(rather than a single cyber defender) as the team undertook the task of defending a simulated network.
The researchers worked with teams of student cyber defenders who were participating in the Mid-Atlantic
Collegiate Cyber Defense Competition. Using a combination of questionnaires and wearable social
sensors, the teams were scored on three measures: services availability, incidence response, and scenario
injects. The researchers showed that team leadership and team interaction were predictors of performance
114

on the scored measures, though performance varied depending on the measure that was scored, suggesting
that team leadership and composition can produce better or worse performance depending on the task that
is to be performed.
External to ARL, there has been growing recognition of the importance of human factors and
usability research in the security area. Interdisciplinary “usable security” researchers work on a wide
range of problems, including understanding the security behaviors of end users, improving the usability of
authentication and access control tools, helping lay people keep their home computers secure, and
improving the usability of security tools for system administrators and cyber security analysts. There is
now a well-established annual conference in usable security (Symposium on Usable Privacy and Security)
and numerous workshops in the area. In addition, the top conferences in security and in human-computer
interaction regularly accept strong usable security research papers. A number of academic departments
around the world are offering courses in this area, and there are now university and corporate research
laboratories dedicated to usable security.
The nascent ARL cybersecurity research effort in the Human Sciences Campaign shows great
promise, but it suffers from not having a critical mass of researchers dedicated to this effort. The group
has done a lot with the resources they have and developed successful collaborations with cybersecurity
researchers in other parts of ARL and externally, but their current staffing level limits both the breadth
and depth of their research.
The ARL cybersecurity effort in the human sciences campaign focuses primarily on studying and
improving the performance of teams of cybersecurity analysts. With the recent construction of the
CHIMERA lab, ARL now has an opportunity to go deeper into this space. The CHIMERA lab is a unique
research facility, and ARL has more ready access to trained cyber analysts than typical academic
laboratories would have. The currently articulated research goals for the CHIMERA lab focus on
developing measurement techniques for assessing the performance of cyber analyst teams and the
technologies they use. The current research is largely observational and exploratory. However, plans are
in place for more quantitative experimental studies. Longer term goals include using data collected from
these studies to inform the design of decision support tools for analysts.
A more ambitious research agenda might be framed around using cyber analysts to improve network
protection. This might include the examination of the performance of individual cyber analysts and teams
of cyber analysts to determine what behaviors lead to the best network protection outcomes and what
human tasks could benefit from automation (using a computer to complete rote tasks without the need for
human intervention) and from decision support (using a computer to assist human tasks and decision
making). This research may begin as exploratory observational work that leads to hypothesis formation.
The CHIMERA lab could then be used to stage controlled experiments to test these hypotheses.
Additional research might examine various approaches to individual and team cyber analyst training to
determine what training approaches lead to the best outcomes. As new tools for cyber analysts are
developed and the Army considers adopting commercial tools, research could investigate the
effectiveness of these tools in improving network protection outcomes.
There are also significant opportunities to conduct cybersecurity research that goes beyond the study
of cyber security analysts and could establish ARL as the preeminent research lab for behavioral
cybersecurity. Current staffing levels limit ARL’s ability to broaden efforts in this area, but additional
staff would allow for a broader research agenda. There are many open research questions in this area that
may be significant to the Army. For example, there are open research questions around how to securely
authenticate personnel and grant access to physical and electronic resources in a variety of real-world
field situations while minimizing disruption of the current task. In addition, a systematic analysis of
security breaches impacting the Army could reveal underlying human-related security problems that
would benefit from a concerted research effort.
115

To meet these opportunities requires increased staffing levels and an interdisciplinary team that
includes cognitive psychologists, human-computer interaction researchers, and security researchers. In
addition, the team would benefit from expertise in machine learning to aid in development of human
behavior prediction models and decision support tools. The team would benefit from interaction with
external researchers at the top usable security, security, and human-computer interaction (HCI)
conferences (e.g., SOUPS, CHI, CSCW, CCS, IEEE S&P, and USENIX Security) and aiming to publish
their research at these conferences. These conferences are useful to attend even when the team does not
have papers to present. The team would also benefit by interacting with other government agencies doing
research in this area, including the National Security Agency (NSA) Science of Security effort.
Real-World Behavior
This initiative is an ambitious and promising entry into the challenging field of the measurement and
analysis of real-world behavior. Overall, the technical quality of the work is high. In particular, the group
has worked to identify technical and theoretical gaps and to align resources to solve specific needs. The
technical quality of enabling technology and instrumentation was especially high. In general, the group
uses strong experimental techniques and appropriate modeling approaches. There has been a continued
improvement in research products, including published papers, chapters, technical reports, and conference
papers. Still, as new research areas are broached, the work would benefit by consultation with appropriate
experts.
Human Variability
The analytical abilities and techniques of the Human Variability program in general are strong. The
EEG-related technical expertise is excellent. The source localization methods being developed are
interesting and a good approach to go beyond simple subtractive methods of analysis.
The Human Variability projects have made progress since the last review by continuing to publish
findings in the scientific literature and present findings at conferences.
Overall, the technical quality of the work is good, and methodologies that are used to explore the
research questions are appropriate.
One area that lacks depth is qualitative research. The work on sociocultural influences in particular
seems heavily dependent upon the use of qualitative methods, but the methods used for conceptualizing
research questions and analyzing and presenting data need some additional expertise to be brought up to
the standards of similar academic research. If this will be an important part of sociocultural work on civil
affairs, then more depth will be needed here.
Another area where more depth is needed is in the area of teams research. In some areas, particularly
the work on teams in cybersecurity, the lack of deep expertise on current literature is leading to slow
progress in the research. The opinion of some ARL researchers is that knowing nothing about the area of
cybersecurity teams reveals a lack of knowledge of other teamwork research. Abstracting the issues of
cybersecurity teams to think in terms of complex, fast-paced decision making in the face of adversarial
pressure would reveal some relevant and useful literature to build upon. The research team appears to be
starting from ground zero in developing its own observations of how teams work together on problems,
and then developing its own frameworks to make sense of observations.
116

A related area that could benefit from more expertise is in multilevel theory and analysis. Ultimately,
to translate the findings across campaigns into actionable conclusions will require integrating findings
from individual-level research into teams and higher levels of analysis. Much of the research presented
was at the individual level of analysis; the few examples of teams research made no use of information on
individual differences, which would undoubtedly affect how teams operate. Simply consulting with
researchers with experience in integrating across levels of analysis is not adequate, because this
perspective is likely to change how the individual level research is designed. Better integration across
levels of analysis will be needed.
Because this line of research is in its infancy, it is very difficult to assess the technical quality of the
work at this time. With that said, it was clear that the researchers are beginning to come up to speed in
cybersecurity as they continue to collaborate with their technical counterparts who have a deeper
understanding of the technical components of the cyber mission. This collaboration will be essential as
the team advances its vision to develop a human science of cybersecurity.
General Conclusions and Recommendations
There is a need across the Human Sciences Campaign to link the science of each project to an overall
theoretical foundation. There needs to be a balance and an identifiable interaction between field studies,
laboratory studies, and the underlying theoretical basis of each. This balance and interaction would be
facilitated by clarifying linkages within projects in a given project area, across projects in the campaign,
and with projects in other ARL campaigns.
Recommendation: The Human Sciences Campaign should develop a clearly articulated plan to
achieve a balance and an identifiable interaction among field studies, laboratory studies, and
the underlying theoretical basis of its experimental and field studies. The plan should include a
strategy for linking the theoretical underpinnings of projects in a given project area, across
projects in the campaign, and with projects in other ARL campaigns, and should identify the
mechanisms for monitoring and guiding formulation and implementation of such linkages,
including, where appropriate, the participation of external advisors.
In a number of important areas across the programs, the in-house expertise is weak. One promising
approach to achieving the necessary levels of expertise and to achieving an effective balance and
interaction between experimentation, field studies, and their theoretical underpinnings would be the
establishment of external advisory committees for each project area, as needed, and, perhaps, across the
Human Sciences Campaign. Other means could include attendance at relevant workshops and
conferences, mentoring by and exchange visits with outside experts, participation in consortia, and
recruiting senior scientists.
Recommendation: The Human Sciences Campaign should consider establishing external

advisory committees for each project area, as needed, and, perhaps, across the Human Sciences
Campaign. Other means should include attendance at relevant workshops and conferences,
mentoring by and exchange visits with outside experts, participation in consortia, and
recruiting senior scientists.
117

There is not a formal program of mentorship and career development within the Human Sciences
Campaign at ARL, and this represents a missed area of opportunity. In particular, new and junior
researchers at ARL need regular career guidance and mentorship from more senior and experienced ARL
scientists and administrators through a formalized program. A program for identifying mentor(s) and
establishing, reviewing, and updating a career development plan would help to develop the research
workforce and ensure that staff members reach their full potential. A mentoring program will not only
benefit the individuals in question, but it will also benefit ARL.
Recommendation: The Human Sciences Campaign should establish a mentorship program for
early-career staff that includes guidance about how to establish a research niche within ARL,
how to choose internal and external collaborators, how to navigate the ARL bureaucracy, when
and how to seek additional training, and other career development activities.
ARL represents a unique setting for researchers. It is well funded, offers exceptional facilities, carries
out interesting and challenging projects, and has numerous partners in academia, industry, and within the
military research arena. These features can make ARL an attractive place for researchers to work,
especially those who are starting their careers. This could make recruiting new expertise easy, but only if
the wider research community is aware of ARL and the opportunities to work there.
Recommendation: In addition to its laudable Open Campus Initiative, ARL should consider
other outreach efforts—especially those targeted at scientists in new areas that ARL seeks to
enter—that may include activities such as booths in exhibit halls at scientific meetings and
circulating specific job or grant opportunities to universities and industry with large programs
in the areas of interest.
Real-World Behavior
The work in the Real-World Behavior program seems narrowly focused, currently, on the behaviors
inherent in driving and marksmanship.
Recommendation: To enhance the generality and richness of its research approaches, the group
should perform an analysis of the range of Army-relevant real-world behaviors and expand its
focus to include study of behaviors additional to those associated with driving and
marksmanship.
Given the centrality of the role of sleep, fatigue, and stress as identified issues in the target real-world
applications and in the military, it would be helpful to find mechanisms such as workshops, scientific
mentoring of outside experts, or exchange visits with outside experts and their laboratories to provide
training opportunities and consulting in these areas, where in-house expertise is not strong. Alternatively,
recruiting one or more senior scientists in these areas could be considered if the centrality of the topic
warrants.
Recommendation: The Real-World Behavior group should consider improving its expertise in
the areas where in-house expertise is not strong—such as sleep, fatigue, and stress— by such
means as attendance at relevant workshops and conferences, mentoring by and exchange visits
with outside experts, participation in consortia, and recruiting senior scientists in these areas.
Recommendation: The Real-World Behavior group should consider developing an advisory

board of scientific experts to provide advice on topics relevant to the major initiatives.
118

To be successful, the Real-world Behavior large experiment projects will require resources and
expertise in the areas of big data, data management, work flow, machine learning, and human-computer
interaction
Recommendation: The Real-World Behavior group should consider reaching out to

information sciences or other computational resources, perhaps within ARL, to look for
opportunities for collaboration in the areas of big data, data management, workflow, machine
learning, and human-computer interaction.
The Refresh program has brought a competitive scientific agenda to new projects and provided
leadership opportunities for mid-level scientists.
Recommendation: The Real-World Behavior group should consider continuing the Refresh
program.
Human Variability
The Human Variability group lacks a framework for entering new research areas or integrating new
aspects of research into experiments. When doing so, it is important to know what you don’t know, have a
plan for seeking expert advice, and select the right experts. This seems to be lacking, and it can lead ARL
to attempt to reinvent the wheel rather than integrating the current science into its ongoing programs. This
theme permeates the following conclusions and recommendations.
It is important to provide opportunities for the researchers to attend scientific meetings and
conferences in scientific disciplines related to new scientific initiatives (e.g., sleep, circadian rhythms, and
genetics). Attending such meetings and conferences will allow the researchers to understand the state of
the science in disciplines outside their areas of expertise, and to network with academic and industry
scientists who are active in these areas. This would also provide ARL with greater visibility within the
scientific community.
Recommendation: The Human Variability group should expand opportunities for the
researchers to attend scientific meetings and conferences in scientific disciplines related to new
scientific initiatives (e.g., sleep, circadian rhythms, and genetics).
There is a strong group of well-trained early-career investigators working on human variability

questions. However, there is a lack of senior scientists working on this effort, and perhaps for this reason
it was difficult to get an overall perspective of how each experiment fits together.
Recommendation: The Human Variability group should seek ways to bring in senior scientific
expertise to coordinate the efforts of the human variability experiments, ensure that duplication
is minimized and overlap is utilized effectively, and mentor the junior scientists working on
human variability and related sciences.
Three scientific areas that are sources of important intra- and interindividual variability are sleep,
circadian rhythmicity, and genetics/genomics.
Recommendation: The scientific areas of sleep, circadian rhythmicity, and genetics/genomics

should be integrated into all ongoing human variability experiments.
Sleep has a pervasive impact on human variability, but the ARL efforts in the area of sleep are
suboptimal. A scientific advisory board comprised of several sleep experts with differing areas of
119

expertise (e.g., collection of sleep data from field settings, impacts of sleep on cognitive performance,
impacts of sleep on physical performance, sleep and shift work/long work hours) could assist in
integrating the measurement and study of sleep in all their studies related to human performance. The
group would benefit by bringing in a senior sleep expert as a visiting scientist/senior fellow to provide
sleep expertise to the ongoing projects (even those that are not yet focusing on sleep) and by collaborating
with a sleep research group that has a broad range of expertise. The Human Variability group could
consider having some of its existing personnel train off-site in leading sleep research laboratories to gain
knowledge in sleep. The group could also consider hiring at least one new postdoctoral fellow with
training in sleep and performance.
Recommendation: The Human Variability group should seek appropriate sleep expertise in the
form of a scientific advisory board, hiring one or more sleep-performance experts and/or
collaborating with a multidisciplinary sleep research group with particular expertise on sleep
recording in field studies, sleep and performance, and individual differences in sleep duration,
need, and response to sleep loss.
Recommendation: The Human Variability group should consider having some of its existing
personnel train off-site in leading sleep research laboratories to gain knowledge in sleep, and
the group should also consider hiring at least one new postdoctoral fellow with training in sleep
and performance.
Circadian rhythmicity (biological time-of-day) is an important source of human variability.
Recommendation: The Human Variability group should incorporate circadian rhythmicity into
all ongoing experiments and should seek to collaborate with experts in the field of human
circadian rhythmicity.
The cutting-edge biomedical research areas of genetics, epigenetics, “-omics” (metabolomics,

proteomics), and biomarkers could provide critically important information about sources of intra- and
interhuman variability, but they are not a focus of any ARL experimentation. It is important that these
cutting-edge areas of biomedical research be integrated into ongoing and new studies as soon as possible
by establishing the right series of collaborations. Senior-level scientific oversight of this effort is needed
to ensure that this is done effectively and across the various studies. This may require a scientific advisory
board or boards.
Recommendation: The cutting-edge biomedical research areas of genetics, epigenetics, “-omics”

(metabolomics, proteomics), and biomarkers should be integrated into ongoing and new studies
as soon as possible, and this should be accomplished by achieving senior-level scientific
oversight, perhaps with the support of scientific advisory boards.
The Humans in Multiagent Systems group needs additional expertise in teams research theory and
methodology. The group also needs additional expertise in qualitative research in order to raise the level
of rigor with which those methods are employed in the sociocultural differences area.
Recommendation: The Humans in Multiagent Systems group should hire another researcher
with deep expertise in teams research theory and methodology and should add expertise in
qualitative methods.
120

The publications of the Humans in Multiagent Systems group were not evident during the review.
The publications would demonstrate research into prior work.
Recommendation: In future reviews, the Humans in Multiagent Systems group should provide
more examples of publications and should provide impact factors of those publications.
Because of the need for research to be Army relevant, project presentations need to identify the study
populations used so that reviewers can evaluate the degree to which findings will generalize to an Army
population.
Recommendation: In future reviews, the Humans in Multiagent Systems group should identify
the study populations used.
Keeping human-agent teaming, cybersecurity, and sociocultural difference research projects in the
same program may be creating areas where necessary depth is lacking in favor of generalists who are
working across areas.
Recommendation: The Humans in Multiagent Systems group should, in collaboration with the
Human Sciences Campaign leadership, examine whether keeping human-agent teaming,
cybersecurity, and sociocultural difference research projects in the same program is facilitating
progress in each of these important areas.
Ensuring that an appropriate proportion of research projects are oriented toward “outside the box”
ideas helps to push the envelope of how Army operations could be carried out from a human and
technology perspective. This could both increase the creativity and impact of the work as well as enable
the group to become more of a force for change within the broader Army organization.
Recommendation: The Humans in Multiagent Systems group should examine its portfolio of
projects and assess whether an appropriate proportion of research projects are oriented toward
“outside the box” ideas.
The researchers did not clearly identify the conceptual and theoretical reasons for the study designs
they are pursuing, nor the ultimate Army-related need they hope to satisfy; these clarifications are
necessary to make clear that they are pursuing research on variables that will have high impact.
Recommendation: The Humans in Multiagent Systems group should clearly identify the
conceptual and theoretical reasons for the study designs it is pursuing and the ultimate Army-
related need it hopes to satisfy.
Enhancing its online presence in providing data, tools, and findings to other researchers would
enhance the Humans in Multiagent Systems group’s scientific leadership in the community.
Recommendation: The Humans in Multiagent Systems group should enhance its online
presence in providing data, tools, and findings to other researchers.
The breadth and depth of expertise in the Human Cyber Performance group is not sufficient to
achieve its stated objectives.
121

Recommendation: The Human Cyber Performance group should increase the number of staff
working in the area of behavioral cybersecurity research, adding both junior and senior team
members with interdisciplinary expertise in cognitive psychology, human-computer interaction,
cybersecurity, and machine learning.
ARL behavioral cybersecurity researchers need to interact with external researchers at the top usable
security, security, and human-computer interface (HCI) conferences (e.g., SOUPS, CHI, CSCW, CCS,
IEEE S&P, and USENIX Security), and need to aim to publish their research at these conferences. These
conferences are useful to attend even when the team does not have papers to present. The team would also
benefit by interacting with other government agencies doing research in this area, including the NSA
Science of Security effort.
Recommendation: The Human Cyber Performance group should interact with external
researchers at the top usable security, security, and HCI conferences (e.g., SOUPS, CHI,
CSCW, CCS, IEEE S&P, and USENIX Security) and should aim to publish its research at these
conferences.
Recommendation: The Human Cyber Performance group should interact with other
government agencies doing research in this area, including the NSA Science of Security effort.
The Human Cyber Performance group does not evince clearly articulated research goals in the
behavioral cybersecurity area.
Recommendation: The Human Cyber Performance group should better articulate the goals of
its research efforts in the behavioral cybersecurity area, focusing on improving security
outcomes.
The Human Cyber Performance group research is largely observational and descriptive; it is not
clearly aimed at providing explanatory and predictive power.
Recommendation: The Human Cyber Performance group behavioral cybersecurity researchers

should think through how to expand their research to go from offering observations and
descriptions to providing explanatory and predictive power.
The breath of the research of the Human Cyber Performance group is limited and does not reflect
clear definition of its research niche vis-à-vis that pursued by academic and university researchers.
Recommendation: The Human Cyber Performance group should expand the breadth of its
research in the behavioral cybersecurity area, emphasizing areas of strategic interest to the
Army that are not receiving much attention by academic and university researchers.
The Human Cyber Performance group would benefit significantly by forming a multidisciplinary
external scientific advisory board that can provide advice and counsel as they start and continue to
advance this new, vitally important scientific endeavor.
Recommendation: The Human Cyber Performance group should consider forming a

multidisciplinary external scientific advisory board that can provide advice and counsel.
122

Analysis and Assessment
The Panel on Assessment and Analysis at the Army Research Laboratory (ARL) conducted a review
on July 11-13, 2017 at the Aberdeen Proving Ground in Maryland. The review was on three of the ARL’s
Analysis and Assessment core campaign enablers—ballistics survivability, vulnerability, and lethality
(BSVL), personnel survivability (PS), and human systems integration (HSI). This chapter provides an
evaluation of that work.
ARL’s Analysis and Assessment (A&A) Campaign provides tools that increase awareness of material
capabilities, assesses the survivability and lethality of Army systems, and both improves and simplifies
the Army’s decision making. The work in the BSVL program provides the analysis and assessment
capability to develop efficient means to understand and influence the factors that reduce vulnerability and
increase lethality of Army ground and air combat systems. The work in the PS program develops and
applies methodologies and tools to model, analyze, and predict effects of weapons against personnel and
soldier protective systems. The work in the HSI program provides methodologies for the assessment of
cognitive and physical human performance trade-offs and workload in support of human systems
integration.
BALLISTICS SURVIVABILITY, VULNERABILITY, AND LETHALITY
The BSVL team is a high-performing group with a wealth of experience and capabilities. The team is
a key contributor to the Army’s system analysis, acquisition, and test and evaluation communities. It has a
large set of responsibilities that it performs to a high level within the limits of the resources provided.
Specific accomplishments that stood out in the work were: (1) improvements in visualization and
augmented reality; (2) computing efficiency efforts; (3) underbody blast advancements; and (4)
automation of manual data collection.
BSVL studies generate a large set of data when assessing various threats, aspect angles, threat ranges,
threat aim points, response variability, and so on. Previously, study results had been presented in difficult
to understand and explain formats. By using the increased computing power and advances in data display
tools, the BSVL team has developed visualization and augmented reality methods that can more
effectively support Army decisions. The demonstration of augmented reality in a ballistic vulnerability
analysis is a good example of applying existing technology to this area to make the outputs more easily
understood by analysts, customers, and decision makers.
The work on profiling the modular UNIX-based vulnerability estimation suite (MUVES) and
distributing the code over parallel processors to gain a factor of 2-8 decrease in computational time is a
significant advancement, particularly as more complexity will have to be added to this code. Two areas
demonstrated significant efficiencies in preparing inputs for the MUVES model—scanning of the systems
to be analyzed in order to automate portions of the building of the geometric target descriptions; and the
123

tablet-based range-data acquisition computer toolkit. These both represent significant reduction in human
labor and make the process more efficient.
BSVL studies are computationally intensive, so historically it typically took weeks or even months to
complete a set of BSVL estimates for an Army customer. Calculations are beginning to take advantage of
the Army high-performance computing capability, which together with advances in parallel computing,
will substantially reduce the time required to perform BSVL studies. For example, high-resolution finite-
element modeling of underbody blast is taking advantage of the high-performance computing resources
so that it can provide a useful tool for designing underbody protection. Once these advances are fully
implemented in MUVES, Army efforts can be completed more quickly and more efficiently. This
advance, once fully implemented, will be a significant improvement, since it will increase the range of
possible outcomes that can be considered in modeling and simulation studies.
The underbody blast modeling effort is a significant advancement. Underbody blasts from improvised
explosive devices (IEDs) are a major concern, and the BSVL team is developing tools to predict damage
and causalities due to underbody blasts. Without a validated and verified tool to predict the effects of
underbody blasts, the Army has been forced to conduct very many underbody blast tests in an attempt to
improve underbody blast protection and reduce casualties. This is a slow and costly process. Once the
emerging tools are verified and validated, they can be put into production. The Army will then be able to
more quickly, economically, and effectively address underbody blast damage.
The work on assessing technologies in development on helicopter blades and technologies that
depend on global positioning system (GPS) data in a denied environment demonstrated the payoff of
conducting these analyses earlier in the acquisition process—that is, “moving left.” Because these had
very different impacts to Army systems, the A&A Campaign needs to work closely with the Assistant
Secretary of the Army for Acquisition, Logistics, and Technology on prioritization of the programs in
order to have the widest impact for the Army.
BSVL studies have relied on wooden witness plates for arena tests and crude plywood mannequins
representing personnel to absorb impacts of fragments, spall, projectiles, and so on for lethality and
vulnerability assessments. The damage in these wood witness damage collectors is measured by hand for
location and by probes for depth, angle of entry, fragment mass, and so on. These measurements are
recorded by hand on data sheets and subsequently entered into computer data bases, again by hand, for
documentation of the results and further BSVL studies. The review team was shown efforts to utilize a
digital “wand” to probe and automatically record data. This offers the possibility to improve the accuracy
and precision of recording results, to reduce the time required and to minimize the potential for manual
data handling errors. This is a worthwhile effort that ought to be implemented as quickly as possible and
expanded to automate as much of the current manual effort as possible.
Recognizing technologies that require new modeling methodologies and having the tools developed
in time to support evaluation is an increasing problem, and the A&A Campaign is falling further behind
in keeping up with the technical and environmental complexities. For example, the work on multihit and
underbody protection modeling, while very worthwhile efforts, were too late to impact current and near-
future designs. A validated model of underbody blast is important to have as soon as possible to avoid
future designs that are very vulnerable to underbody blasts. As another example where the A&A
Campaign is behind, in order to lessen combat vehicle weight, ceramic armors have been proposed as a
replacement for metal-based armors, but ceramic armor is very sensitive to multihit damage. The A&A
Campaign is just now initiating efforts to develop methodologies and tools to address multihit damage to
ceramic armor. These armor technologies were developed for the Future Concepts Systems more than 10
years ago.
There is a need for the Analyses and Assessment Campaign to move to multithreat analyses. For
example, an active protection system can have targeting and tracking sensors on a vehicle that is being
124

protected. These are vulnerable to electronic warfare and cyber attack. Hence, an analysis of the
effectiveness of the system requires interdependent analysis of electronics, cyber, and ballistic
interactions. Other technologies where electronic and ballistic interactions are not independent include
robotics, manned and unmanned teaming systems, and smart munitions. It will be important to analyze
these as a coupled system.
Not only are the technologies increasing in complexity, but the environment in which the Army is
fighting and may fight in the future include complex urban environments with a crowded electromagnetic
environment, variable atmospheric conditions, and close proximity between the enemy and
noncombatants. The electromagnetic environment can be generated only in computer simulations, as there
is no comparable environment where the United States can do open-air tests. These added complexities
will need to drive changes in the methodology used to produce ballistic survivability and lethality
information, and multithreat analyses. There is a need to include multithreats (ballistic, electromagnetic,
and cyber) and complex environments in ballistic survivability and lethality analyses. Currently, such
information is averaged too early to be most useful in subsequent studies. The increases in complexity
will drive a need to take full advantage of the DOD high-performance computing (HPC) resources.
At present, there is a lack of active duty military personnel in the BSVL team. Military experience
and expertise is vital in developing study plans and understanding results in a military combat context so
that efforts better relate to the military environment; and the impact of results on the military situation can
be more readily understood by the Army community. This lack of military personnel impacts the value of
the BSVL team’s efforts for the Army.
The BSVL team has been reduced in personnel by one-third from the peak of a few years ago.
Nevertheless, there continues to be a significant volume of study requests by the Army community to
address as well as significant tool improvement and development required on current and emerging
systems, technologies, threats, and so on.
While the efforts being introduced to address critical technology, environment, and threat changes in
areas such as manned-unmanned teaming, active protection systems, GPS-denied environments,
multidomain integration, behind-armor blunt trauma (BABT), multihit damage/effects, and so on are a
positive development, most (if not all) of these concerns have been known for substantial periods of time.
The BSVL team needs to stay current with ongoing changes and address emerging and new technologies
and threats with new and modified tools to address emerging challenges. The efforts in the past have not
always been successful in keeping up as required to support the Army community needs.
The use of HPC resources has improved the BSVL team efforts. However, there are opportunities to
make even more extensive use of HPC resources to improve BSVL tools, productivity, and overall
contributions to the Army community they support. Using HPC resources to a much greater extent would
have a multiplying effect on the overall contributions to the Army community and on how BSVL tools
are utilized.
While the efforts on data acquisition are an important first step, methods to fully automate this
process and eliminate touch-labor are important and needed. There may be high-resolution imaging
techniques to gain all the information required.
While the review team noted some accomplishments and advancements in computing for displaying
data and increasing speed of results, no mention was made of any effort to address a basic factor in the
whole BSVL approach. Shot lines are modeled as mathematical nondimensional straight line rays, while
physical projectiles, fragments, jets, and so on obviously have three-dimensional (3D) shapes that interact
with armor, components, fluids, and crew. While it is not fully understood what the impact this
simplifying assumption might have on BSVL studies, this deviation of the model from reality warrants
some thought, as it is a long-standing concern about a basic tenet of the BSVL tools.
125

PERSONNEL SURVIVABILITY
In the ARL A&A activities in personnel survivability, there was a strong focus on performing
analyses at the earliest suitable time during the life cycle of programs, early enough that positive or
adverse findings and analysis results could be used to inform and to improve decision making. There was
clear understanding that this “move to the left” is a trade-off between informing programs at an early
stage and multiple, potentially costly analyses for programs prior to major decision points. The
implication is that “move to the left” does not necessarily imply the need for early-stage analysis to be
performed for every program. The choice depends on cost and program importance.
Statistical analyses underpin most or all of ARL’s A&A missions. In light of the importance of
statistics, the presentations showed a laudable emphasis on statistical concepts beyond simple means that
are often used in military programs, and need to understand confidence intervals and variance in the
analysis of programs. As a principal part of statistical analyses, the presentations especially emphasized
proper selection of appropriate analyses. This is of significance, since ordinal data has been misused as
integer-continuous data in engineering statistical analyses. The ARL team clearly recognizes this issue
(Abbreviated Injury Scale [AIS] 2 is in no sense twice as severe as an AIS 1) and specifically underlined
the use of the rich field of nonparametric statistics for ordinal data.
Behind-armor blunt trauma (BABT) and behind-helmet blunt trauma injury risk assessments are long-
standing and urgently needed capabilities both for fundamental research and in analysis and assessment
missions. This is of crucial importance for informing weight trade-offs with deforming passive
momentum defeat mechanism personal protection in both the head and torso. The current focus on BABT
injuries from the helmet is commendable and appears to be strongly supported. The team has early-career
researchers who are engaged, and are knowledgeable about previous limited work in the field. ARL has
taken a good approach for the physiological experimental work combining high-speed X-ray imaging
with state-of-the-art and exploratory sensor technologies. This is an example of outstanding work in an
area where there has been an urgent need for over a decade. This work ought to continue and be extended
to thoracic BABT, for mid-chest (mediastinal), abdominal, and spinal impacts.
An additional area of commendable progress that supports fundamental analysis of personnel is
ARL’s development of capabilities for imaging and ongoing generation of human model data, particularly
a range of anthropometry data from medical imaging. Promising aspects include acquisition of imaging
data from living people. In addition, presentations outlined the development of dynamic surface imaging
for occupational and personnel survivability assessments inside vehicles, using commercial 3D imaging
systems. Full development and use of this capability can provide additional valuable information in
assessing dynamic positioning for both use assessments and realistic dynamic locations for vulnerability
assessments. There is a continuing effort to strengthen and develop high-speed flash X-ray imaging for
dynamic blast and ballistic assessments; this is an important tool for assessment of injury from physical
models to physiological models, providing valuable feedback on the high-rate dynamic response of
rapidly deforming personal protection.
The ongoing development of enabling technology for detailed and realistic finite element models of
humans for personnel vulnerability assessments is important. These efforts include good use of high-
performance computing support from the core supercomputing facilities of the army. These efforts
include support for fundamental human tissue property characterization building on previous work both at
ARL and in academia. Efforts in this direction need to be actively encouraged for both mid-term and
near-term research supporting A&A efforts. Such efforts are enabling for future generations of high-
fidelity models, which may support substantial improvements in both vehicle and personal protection.
For biomedical research, the personal survivability group has good, modern facilities. These include
live-fire ranges for explosives, fragmenting weapons, ballistics, and available medical assessments of
various types. The facilities include a mobile clinical computed tomography (CT) scanner, available
computational facilities include substantial Army supercomputer resources. In recent years, ARL has
126

developed the ability to perform needed biological experimentation using various physiological and
cadaveric models. In parallel with the near- and mid-term development of analysis tools using these
models, ARL continues to refine simple physical models such as efforts to replace existing plywood
dummies with more anthropomorphic and physically responding manikins. Progress in this area needs to
be maintained and improved with core efforts and collaborative efforts among potential research partners
to support A&A efforts.
The principal opportunities and challenges arise with the essential tools of ARL’s A&A Campaign—
MUVES and operational-requirements-based casualty assessment (ORCA). Both are indispensable, but
both need to be further developed and validated in several areas over a near-term to long-term time frame.
For MUVES, validation and extension efforts are clear and critical; no other organization will develop a
robust analysis tool central to the ARL mission. For ORCA, though long-needed efforts have begun to
improve basic aspects of components of the framework, principally increasing resolution for the
ComputerMan component, planning and core funding for additional improvements and validation are
urgently needed. It is important that the development of these tools does not occur solely on an ad hoc
basis when responding to immediate A&A tasks. The scope of these tools for Army development and
planning is so broad that development needs to occur on a deliberate and well-funded basis. Long-term
tool development of both MUVES and ORCA have similar issues, the priorities are not clear on any time
frame beyond current development. The challenge is to develop an organic plan for long-term
development of both tools that incorporates current threats and addresses risk assessments for threats to
personnel in the intermediate and long term. Desirable developments include increases in computational
efficiency, especially for parallel use. Much of the use of ORCA and MUVES are essentially fully
parallel assessments. Some of this development has started. In addition, the provision for closer and more
organic integration of the individual tools within ORCA would be highly desirable. It would also be
extremely valuable to add BABT capabilities for personal protective equipment (PPE), particularly
helmet and thoracic body armor. It is essential that ARL provide consistent long-term planning and
funding for this effort.
Another key area of challenges and opportunities for personnel vulnerability A&A is in the medical
arena. Building on a substantially improved collection of battlefield data by Joint Trauma Analysis and
Prevention of Injury in Combat (JTAPIC) and the Joint Trauma Program, ARL analysts have the potential
for supporting substantial near-term, mid-term, and long-term improvements in survivability and injury
risk assessments. However, substantial challenges impede the most efficient use of this data. Key among
these challenges is the difficulty of obtaining detailed medical data (principally from JTAPIC and U.S.
Army Medical Research and Materiel Command [MRMC]) beyond the coded injury descriptions
provided in redacted form, apparently based on the incorrect assumption by the medical community that
detailed medical information is not useful for engineering survivability risk assessments. This is an
essential limitation (or domain conflict), since effective personnel risk assessments cannot be performed
without granular knowledge of injuries well beyond AIS coding categories. Indeed, such detailed medical
information offers the opportunity to inform and develop more effective risk models based on battlefield
functional capacity, rather than simple risk of injury or fatality reflected in the AIS scores alone.
A related challenge for ARL in developing appropriately granular personal vulnerability A&A for
military programs is the continued use of injury coding schemes such as AIS to classify injuries. In this
system, injuries are ranked using an ordinal scale from 1 (minor injuries) to 6 (maximum). This scaling,
developed principally as a threat to life scale in the automobile biomedical community, has been extended
to military injuries. However, this extension cannot fully remediate the problem with its use in the
military domain; threat to life is one potentially important aspect in program A&A, but is not necessarily
paramount. For example, a survivable mid-shaft femur fracture (AIS 3) may be as militarily significant as
a likely fatal injury from an aortic laceration (AIS 5), depending on the needed functional capacity for the
127

mission. For future efforts, it is important to both collect and assess injury data with higher granularity
and with an eye toward the use of military injury data as an assessment of the mission-oriented functional
capacity on the battlefield. The benefits of using additional available battlefield injury data for various
threats spans both the development of research tools for A&A and their use in various operational
domains.
One long-standing challenge is in the development of models for BABT, especially for the head and
torso. This development includes the refinement or replacement of existing physical models for ballistic
BABT with various fragments and ballistic threats. This subject has been addressed by several previous
National Academies studies, and the essential recommendations have not been addressed even after the
passage of a number of years. These previous National Academies studies emphasize the limitations of
both the physical models and the underlying injury biomechanics. For example, the expedient “Prather
model” for ballistic BABT injury applied to soft VIP body armor for handgun threats in the 1970s was
never intended to apply to rifle-round ballistic threats behind hard body armor. As noted earlier, laudable
investigations of helmet BABT in physiological models has begun, but the development of such models
and physical surrogates has been needed from the beginning of the development of aramid composites for
ballistic helmets in the 1980s and the development of hard body armor in the 1990s. Owing to the
importance of such models to the Army, addressing this challenge needs to take a central position in the
development of personal vulnerability models. It is important for these models to include a plausible set
of injury assessments with various battlefield functional capacities and universal joint tasks, not just
lethality and severe incapacitation models.
Principal issues in the development of such models include “closing the loop” to validate
structure/physiology results derived in physiological models in humans. This effort needs to be two-
pronged; both computational models of humans and surrogates and field epidemiology play a role. For
computational models, the challenge of improving finite element models can be addressed, in part, by
attempting to add models that contractors develop for military programs, paid for by the Army, to the mix
of available tools for the broad development of useful BABT injury risk models. For field epidemiology,
collaborations with Program Executive Office Soldier, MRMC, the Armed Forces Medical Examiner, the
intelligence community, operational commands, and others need to be formalized to obtain as much
information from existing personal protection, including vehicles and armor to assist in developing and
validating risk models. For example, personal protective equipment (PPE) needs to be sent back to ARL
from theatre both in instances where it has worked successfully and in instances where the soldier has
sustained injury in spite of (or because of) the PPE.
The ARL A&A Campaign has focused on using and developing tools based on appropriate statistical
analyses, including the assessment of variance and distributions in calculations rather than the use of
simple means. Essentially every aspect of the A&A mission involves statistical distribution rather than a
fixed single exemplar. For instance, threats have a distribution of energies and effects. Human
anthropometry arises in military operations as a distribution. Response of vehicles and personal protective
equipment can be characterized by a distribution. Even material properties of constituents of vehicles,
protective equipment, and people are a distribution. The challenge of using appropriate statistics in
assessments is twofold. Often the distribution needed for a given analysis is unknown and is not feasible
to measure or obtain. So, approximations need to be made to make the assessment tractable. The other
major challenge is determining the appropriate analysis for significance. In the medical and biomedical
communities, it has recently become even clearer that the often-used statistical significance value (the “p
value”) may not be a desirable or effective measure of the import of a statistical difference. That is, the
simple statistical significance does not tell one whether the difference is “important” or not. If the
variance of a particular measurement is small, even unimportant differences can be statistically
significant. When assessing the significance of an effect, it is important to consider not only the statistical
significance but also to consider the size of the effect and whether a difference is a clinically or
programmatically meaningful difference. Evolution of this philosophy continues in the biomedical field to
emphasize important differences in treatments, not simply differences with a statistically discernable
mean.
128

There appears to be insufficient core funding to advance key areas in BABT risk modeling, and the
existing efforts appear to be currently funded on a somewhat ad hoc basis.
HUMAN SYSTEMS INTEGRATION
The ARL HSI team has beneficially applied the Improved Performance Research Integration Tool
(IMPRINT) and digital human modeling to Army system concepts prior to contracting with industry to
fully develop new weapon systems. This has led to early design improvements and has avoided costly
redesigns later in the development cycle. Following pre-Milestone B applications, these models have been
provided to system development contractors to continue iterative HSI analysis as system concepts and
designs continued to evolve. The HSI team also developed a much-needed Manpower Requirements
Criteria (MARC) toolset that enables designers to trade off candidate designs to cost-effectively optimize
soldier accommodation.
The ARL Human Research and Engineering Directorate (HRED) has recruited and hired early-career
scientists and engineers and is mentoring them to fill in for more experienced personnel who will be
retiring in the near future. This contributes positively to ARL’s ability to provide continuous support to
the development and maintenance of their HSI tools, models, techniques, and methods.
There has been much work, over many years, to improve and extend the capabilities of the digital
human models and the accompanying soldier clothing and equipment models. This has allowed
comprehensive static evaluation analysis and assessment of male and female soldier accommodation in
ground combat vehicles and other systems. ARL supports contractors in the use of these models during
design development to ensure their proper employment. The extension of modeling analysis techniques to
include dynamic conditions is applauded, as it could significantly improve the ability to analyze vehicle
safety.
The ARL HSI team developed, improved, and used the IMPRINT model to predict soldier
performance and workload in Army systems for nearly two decades. IMPRINT has been employed by
hundreds of contractors to assess system concepts, preliminary designs, baseline designs and to propose
design changes. This has resulted in significant cost avoidance over many years.
The Job Assessment Software System (JASS) tool is being developed to compare the skills and
aptitudes required for Army jobs against existing military occupational specialty (MOS) requirements.
This tool has potential to assist contractors in assessing soldier interface characteristics and job demands
against the capabilities of the personnel identified in the target audience description (TAD) for
operational, maintenance, support, and training positions. This tool, if appropriately verified and
validated, could help to identify areas where jobs have been designed that do not match the capabilities of
the soldiers identified for those jobs. It then may also allow contractors to correct user interfaces, modify
job or training requirements, or propose changes to personnel requirements prior to establishing the
system baseline at the stage of preliminary design review (PDR).
Opportunities to involve military personnel as subject matter experts (SMEs), subjects, and assistance
with HSI tool/model validation ought to be identified. More direct interaction with warfighters is essential
to provide high confidence that real Army problems are being successfully addressed in a proactive,
timely, and efficient manner. The mechanism for acquiring soldier assistance needs to be more formalized
and not inconsistent and ad hoc as it currently appears to be. Without good ARL situation awareness (SA)
with respect to the real needs of the soldier and a continuous focus and feedback loop to ensure that the
solutions are usable, there is a high risk that the mission will not be properly supported; there will be
129

much resulting loss of life and material, and a failure to achieve objectives. As an absolute minimum,
ARL HSI principal investigators (PIs) need to take advantage of opportunities (and be encouraged) to
spend time with soldiers in the field to gain an appreciation for their tasking, operational environments,
risks, hardships, and the trade-offs that need to be made on a daily basis.
An overarching framework for HSI analysis and assessments ought to be defined and implemented.
Needs from potential users (including the Army, contractors, Federally Funded Research and
Development Centers [FFRDCs], academia, etc.) ought to be solicited to identify gaps, prioritize them,
and use the results to guide future analysis and assessment investments. This would provide a clear
rationale for each A&A tool, technique, and method in terms of Army customer needs and provide the
information necessary for a coherent evidence-based prioritization for application of resources. It is not
clear why A&A tools that have been emphasized and used on past Army acquisition programs (such as
goal-directed task analysis [GDTA, IMPRINT, and SA analysis) are not being aggressively maintained
and applied. Other tools (e.g., Preventative Maintenance Checks and Services [PMCS+]), while reducing
task completion time and errors, do not appear to add new analysis or assessment capabilities. An
overarching framework, that includes all HSI domains and relates the A&A techniques to Army needs, is
needed to identify areas where stakeholders are being under-served.
It is very important for the operational requirements-based casualty assessment (ORCA) model to
accommodate various sizes and shapes of female and male soldiers to provide more useful results to guide
program-level decisions. The current version of ORCA uses a 50th percentile male digital human model
for all calculations. Creation of several different female and male models of varying anthropometry would
significantly enhance the ability of ORCA to provide accurate analytical results.
Rigorous verification and validation (V&V) ought to be viewed as an essential step in HSI tool and
model development. Warfighter participation is needed to validate the effectiveness of HSI A&A tools,
models, and techniques because they are intimately familiar with the combat environment in which
soldiers must perform their mission tasks. Failure to rigorously validate A&A tools in a timely manner
throughout the entire cycle increases the likelihood of inefficient use and loss of personnel and material
resources to successfully accomplish mission objectives.
More emphasis needs to be given to the transition of tools, operator manuals, training, accompanying
approaches and methods to industry. Many of the models, tools, techniques and methods developed by
HSI could be cost-effectively applied by system development contractors as they iteratively define, refine,
and baseline system designs. To facilitate this, ARL could provide contractors with HSI models and tools,
operating manuals, and training. ARL HSI scientists need to also establish a professional relationship
with contractor personnel operating these tools, models, and so on to gather lessons learned and ideas for
tool and model enhancement.
ARL ought to focus more on using the results of analyses and assessments to improve human-system
integration requirements for future acquisition programs. This could take the form of improved contract
requirements or upgraded HSI domain standards. The current approach reflected in the human modeling
area, is to evaluate contractor designs post-Milestone B. Delaying evaluation until post-Milestone B may
be too late in the acquisition process to make cost-effective design changes. It is suggested that improved
accommodation or soldier “space claim” requirements could be developed for future contracts, based on
ARL’s 10+ years of Jack modeling experience. Improved requirements could lead to earlier problem
detection and resolution.
It is important for ARL to develop a capability for analyzing and assessing HSI (e.g., human factors
engineering [HFE], soldier interface and training) technologies for their potential in improving human-
system integration and performance. One of the areas of emphasis for the A&A Campaign is titled “A&A
on Technologies.” Currently, this area is limited to “SLV A&A of Technologies” and “Technology Trade
Space A&A.” “Human Systems Integration Technologies” need to be added as a third subarea. Emerging
soldier interface technologies need to be analyzed to determine if they hold potential for application to
future Army systems. Additionally, formal usability assessment tools ought to be developed for analysis
of soldier interface technologies.
130

JASS may hold potential for broader applications. Contractor HFE and user interface designers
usually do not have knowledge of the soldier occupational specialties and grades that will be operating
their systems to accomplish their required mission tasks. Without knowledge of the skills, aptitudes and
knowledge of the users, contractors depend upon retired military or surrogate soldiers to analyze and
assess their designs. With modification, JASS might be used as a tool to evaluate soldier jobs and user
interfaces against the military occupational specialty (MOS) requirements for that position as established
by the target audience description (TAD). This could result in earlier detection of mismatches between
soldier capabilities and job requirements.
It is important that anthropometric models that deal with more realistic scenarios and dynamic
conditions for soldier protection be developed soon. ARL presented plans to develop digital human
models capable of analyzing varied scenarios and conditions that cannot be analyzed today. In addition,
plans were put forward to extend accommodation analysis to include the dynamic conditions that vehicle
occupants may experience—for example, rough terrain and improvised explosive device (IED)
detonations. Including realistic, dynamic conditions in physical accommodation analysis could
dramatically improve the validity of results and allow a higher probability of overall mission success; it
could also materially improve soldier safety in the operational environment.
Some popular and important tools and models (e.g., IMPRINT) are not being supported as strongly as
in the past. Improvement plans for other A&A capabilities (e.g., digital human modeling, dynamic
accommodation) seem to extend over an excessive time span. A capability to perform dynamic soldier
accommodation modeling, for example, is long overdue and compromises the ability to appropriately
design vehicles and execute missions in an optimal manner.
The number and scope of analyses and assessments need to be expanded to address all domains of
HSI. The analyses and assessments presented did not address all of the HSI domains. ARL ought to
proactively collaborate between domains to ensure that all aspects of soldier integration are addressed.
Currently, the ARL A&A scope emphasizes physical and cognitive soldier accommodation. To extend
this, ARL/HRED can “reach out” to other Army organizations (e.g., Survivability and Lethality Analysis
Directorate [SLAD] for force protection and survivability and ARL Orlando for training) and agencies
responsible for the other HSI domains (i.e., habitability, safety and occupational health) and include their
A&A tools, techniques and methods in the overall HSI toolset.
The ARLTAB assessment of this campaign is different from that of most, if not all, other campaign
assessments, since the Analysis and Assessment Campaign is intended to be more of an analytically
focused, crosscutting activity rather than being research focused. As a result, the criteria are different
from those of research-focused campaigns. Nevertheless, the work needs to have technical depth.
The quality of the technical staff and the quality of the work reviewed was generally outstanding.
Nevertheless, current A&A efforts are falling further behind in incorporating the complexities of the
technologies and environments needed for A&A. This could be due to lack of resources—either financial,
personnel, or both. In several areas, the team is only one deep, so there is a lack of personnel, likely
driven by funding. Resources need to be made available to address the requirements to analyze and assess
complex technologies in complex environments.
The modeling work is generally of high quality, but rigorous verification and validation is not always
included in model and tool development; this is especially the case in HSI.
Of the core technical BSVL efforts reviewed, the projects on underbody blast modeling and the
collection of data that shows the impact of multiple hits on armor panels were of very high technical
quality, and the teams were highly educated and skilled to conduct these efforts. However, limited
resources, either funding or personnel, resulted in important effects being modeled well after this
information was needed. A lack of validated models has led to the need to carry out a very costly
experimental program illustrating that it is important to stay ahead of the need.
131

Three of the Key Campaign Initiatives (KCIs), “Framework for Complex Multidomain Analysis,”
“Analysis & Assessment of Congested & Contested Operational Environments,” and “CEMA Analysis
and Assessment Methodology for Congested and Contested Environments,” have significant overlap, are
well outside the current mission space of ARL, and are so broad that the outcomes cannot be clearly seen
in the 15-year time frame. More definition and work is needed on these initiatives. The fourth KCI,
“Virtual Interactive Simulation Analysis and Assessment” is a long-term follow-on effort from the
immersive demonstration that served as a proof-of-principle for these visualization techniques. These KCI
efforts would all benefit with near- and long-term deliverables defined.
The area of analysis and assessment is a very important area to the Army. The analyses and
assessments prepared by ARL support Army decisions at all levels of the Army. The products of these
assessments and analyses are used by the Army Evaluation Command in preparing recommendations to
Army leadership up to and at the Secretariat level. A&A is an important activity that is very under-
resourced and falling further behind in meeting mission goals, which puts ARL at risk to losing this
activity to another Army organization. It is important to keep this activity in ARL to link the 6.2 tool
development with 6.6 tool application.
Military experience and expertise is vital in developing study plans and understanding results in a
military combat context so that efforts can relate better to the military environment and the impact of
results on the military situation can be more readily understood by the Army community.
There is a potential problem when a contractor or 5-year term employee is the only employee
possessing a critical skill for a large project of major importance for the Army.
In some cases, government employees need to use contractors and contractor facilities to get work
done that they cannot do because of the work environment and lack of adequate computational support.
The following recommendations are offered.
Recommendation: ARL should prioritize tool development to reflect current and future Army
acquisition needs and provide longer-term projects with predictable funding.
 ARL should develop and articulate clear long-term priorities.
 ARL should provide long-term sustainment and high-priority improvements to
core analysis tools (MUVES, ORCA).
 MUVES and ORCA should be examined to see if they are structured adequately
to handle multihit, multithreat, and complex environment interactions in the
future.
 ARL should include plans for verification and validation as an integral activity
throughout the tool development process; this is especially the case for HSI.
 ARL should consider more sophisticated frameworks (sort of a grand
challenge), with a focus on creating the right team, with industrial and academic
partners.
 ARL should consider a red team/blue team approach for analyses/assessments.
Recommendation: ARL should increase engagement with military personnel and program
managers (PMs).
 ARL should increase involvement of military personnel in the prioritization
process and in tool and model verification and validation.
 ARL should involve PMs at an early stage of the prioritization and development
process and in the contracting language for A&A support.
132

 Engagement with other services, NATO, TTCP, and other military partners
should be further improved with benefits to the Army.
Recommendation: ARL should review the use of contractors and 5-year term employees.
Recommendation: ARL should improve the work environment for A&A staff.
 ARL should speed up the approval process for conferences and equipment
purchases and have these take place a lower level.
 Information technology (IT) support for A&A staff should be improved.
Recommendation: ARL A&A Campaign should develop and lead only centers well related to
the scope of A&A activities and responsibilities within ARL and for which the ARL A&A
Campaign leads. For activities that ARL A&A Campaign does not lead, ARL A&A should
consider joining existing centers.
Recommendation: ARL should develop increased interactions between the A&A Campaign and
various relevant 6.1 and 6.2 efforts within ARL.
133

Crosscutting Conclusions and Recommendations and Exceptional

Accomplishments
CROSSCUTTING CONCLUSIONS AND RECOMMENDATIONS
The following crosscutting conclusions, recommendations, and exceptional accomplishments are

based on the projects and programs presented, as a full spectrum of projects and programs within each
Army Research Laboratory (ARL) campaign and the interrelating mapping across all campaigns’ projects
and programs were not provided to the Army Research Laboratory Technical Assessment Board
(ARLTAB).
Research Positioning
To be successful and productive, research needs to be properly positioned with respect to the broader
scientific community including industry. Such positioning requires cognizance of the objectives of
relevant research efforts, an analysis of the strengths and weaknesses relative to this broader scientific
community, and a plan to develop the critical knowledge and expertise needed to best achieve program
objectives.
Recommendation 1: Upon initiation, ARL research efforts should propose a positioning plan
and schedule that includes
1. Identification of key, core, and complementary research programs and relevant
expertise;
2. A statement of ARL’s intended role in the context of this expertise (lead, follow,
support); and
3. An appraisal of the need for external and internal technical support—for example,
via external advisory boards, visiting researchers, workshops, or collaborations—
and if needed a plan to develop such support.
Leveraging Scientific Talent
As research problems have evolved to include multidisciplinary interactions, there has been a
concomitant growth in collaboration among researchers, both within ARL and with those from the larger
external community. In this research environment, it is important to acknowledge that common
foundational technical skills may exist in ARL in different campaign thrusts, where they are being applied
in uniquely different ways. Knowledge of the existence of these technical skills would be highly
beneficial, both from the perspective of bringing greater technical talent to bear on existing problems, as
well as promoting synergy among campaign thrusts by bringing to light possible new areas of
collaboration. There was some evidence, for example, that system intelligence and intelligent systems
134

(SIIS) researchers within ARL were not always aware of similar or parallel efforts in other campaigns. In
looking at research related to human-machine or human-information interaction, it is clear that individual
projects within campaign thrusts would benefit if collaborations with researchers with parallel or
complementary skills in other campaigns could be facilitated. A systematic mapping of primary and
secondary skill sets of the technical staff at ARL would be highly beneficial in this context.
Recommendation 2: ARL should bring about greater understanding of available technical

expertise in critical subject areas across ARL, and leverage this expertise to build greater
synergy across campaign thrusts.
Predictable Funding
Multiyear research projects that undergird the development of future Army capabilities rely on the
development of experimental tools and modeling capabilities that require years to develop. Predictable
funding levels are needed over the project lifetime (with, of course, milestones for continued support) in
order to plan an effective strategy for acquisition of equipment and the hiring and training of personnel.
Recommendation 3: ARL should provide predictable long-term funding for multiyear projects.
Streamlining Research-Related Approvals
ARL staff reported that the time period for the approval process for conference attendance and for
equipment purchases is often too long. Attendance and presentations at national and international
conferences play an important role in bringing an awareness of the latest developments in a field into the
purview of ARL. They also play a major role in making the outside world aware of the high-quality
research at ARL, thereby enhancing the reputation of ARL and demonstrating to young scientists and
engineers that a career at ARL would be attractive. It is important that procedures for attendance at such
conferences should not make preparation for participation a chore and thereby discourage ARL staff from
pursuing such opportunities.
Incidents occur in the course of carrying out research where an additional piece of equipment is
needed or where computer software or hardware needs to be purchased to complete a project. In some
cases, because of the time delay in acquiring what is needed, government employees need to use
contractors and contractor facilities to get work done that they cannot do because of such delays or
because of the lack of adequate computational support. Procedures need to be in place for reducing the
time scale for justifying the need and then obtaining what is needed as well as providing timely
computational support when needed. An expedited approval process for conference participation and
equipment purchases would enhance staff productivity and ARL staff morale.
Recommendation 4: To facilitate research, ARL should streamline the approval process for
conference participation and procurement of equipment.
Systems Approach
Effective research assessment processes require (1) clear articulation of the expected outcomes for
each portfolio, including key technical milestones and metrics to be used for measuring progress and
success; (2) definition of an appropriate set of outcomes and metrics for each project within a research
portfolio; and (3) a closing of the loop that documents and describes the actions resulting from the
assessment process. Beyond providing researchers the ability to gauge progress and make midcourse
135

adjustments (which may in some cases arise from an unanticipated discovery), including a refocusing of
portfolios and projects determined to be too broad in scope, a well-structured assessment process provides
greater visibility to collaborators and partners. This is especially needed for research that entails a systems
engineering approach.
Every Army system is intended to serve, enable, and protect the soldier within the system. Army
systems that do not interact with the soldier at some level do not exist.
Recommendation 5: ARL should place greater emphasis and focus on a systematic assessment
of its research. The assessment should include measureable milestones, outcomes, and metrics
for the portfolios and the projects within them. In all ARL campaigns, research efforts aimed at
developing any system should endeavor to understand, incorporate, and accommodate the
soldier within the system through the incorporation of human systems integration (HSI)
principles. HSI should include the consideration of usability, sustainability, resilience, and
survivability within the system.
Stewardship of Data
The explosive growth of data—observational, experimental, and computational—and the rapid and
concurrent development of new machine learning techniques is opening new opportunities to extract
insights and patterns and create predictive models of natural phenomena and human behavior, while also
preserving unique or historical context and experiences. Following theory, experiment, and computational
modeling, big data analytics and deep learning have been called the “fourth paradigm” of scientific
discovery. ARL is well positioned to host unique Army-relevant data and metadata and to engage the
research and industry communities in collaborative partnerships. Equally importantly, such hosting and
engagement can accelerate ARL research by applying data analytics and machine learning across
disciplinary and multidisciplinary contexts, while also attracting new talent and ideas.
Recommendation 6: ARL should develop and host a curated data repository of select Army-
relevant data, targeting domains and contexts relevant to its strategic objectives and preserving
data and contexts that may otherwise be lost. In conjunction with development of the data
repository, ARL should develop a set of Army-specific data analytics questions and sponsor
competitions to accelerate progress on ARL problems and attract new talent and expertise.
Enhancement of Approaches to Inform Theory
In order for ARL research to accelerate and to produce useful and meaningful findings, the research
needs to be approached in a systematic manner that includes the consideration of environmental
conditions, an understanding of system response to expected stimuli, and an understanding of the overall
behavior of the system under consideration. Given this context, ARL research efforts need to consider this
broad systems research approach to develop and enhance research efforts. The ARL observational and
experimental research endeavors seem generally adequate, but there appears to be a weakness in the
theoretical underpinning of the research in some areas. Such weakness in underlying theory has been
addressed by modeling.
ARL researchers demonstrated general awareness of the facts that in the conception, design,
implementation, and assessment of scientific solutions of technical problems, the use of models
commands a central and critical position in the development of technology, and that considering the
complexity of phenomena of interest to ARL, useful modeling provides a means of advancing the
development of technology. In some cases, however, there appeared to be incomplete appreciation of the
fact that useful models incorporate complexity, scalability, robustness, uncertainty, and operations in
136

noise and interference. The uncertainty in such modeling may be large. Applying state-of-the-art tools to
analyze uncertainty, ARL can develop and use models to advance technology.
Recommendation 7: The ARL research efforts within a particular campaign should comprise
four components: (1) real-world observations (for example, surveillance, field research, and
naturalistic observations); (2) laboratory testing; (3) theoretical underpinning of the science
(for example, modeling and simulation); and (4) assessment, verification and validation, and
uncertainty quantification of the models. All research should endeavor to contribute to one or
more of these research components in such a way that each component’s findings serve to
inform the other research components. In addition, a balance should be met between the
contributions to these various components so that an overall systems appreciation is achieved.
ARL should further enhance the use of appropriate models to better understand the
phenomena of interest and develop technology.
Enriching Open Campus
Research software is rapidly developing, and qualification of software for use on internal ARL
networks takes time. Staff members reported that this situation forces them to choose between delaying
using software that facilitates their research or, if they can, going to contractor (or other) offsite computer
facilities to carry out their work. This delays or in extreme cases can preclude, carrying out promising
research projects.
Recommendation 8: To enrich the ARL open campus, ARL should consider developing an ARL
on-site open network that research staff can use to readily access research software that has not
yet received qualification for use on the internal network.
EXCEPTIONAL ACCOMPLISHMENTS
The following are the exceptional accomplishments for each campaign area.
Materials Research
The work on high-voltage aqueous electrolytes could be revolutionary for battery technology for the
Army and elsewhere. Work on radioisotope-based power sources is also noteworthy, having progressed
very rapidly from concept to implementation. This work is considered by the ARLTAB to have
significant upward potential.
In just a few years, the quantum sciences program has attracted outstanding investigators, driven in
part by their membership with the University of Maryland and the National Institute of Standards and
Technology (NIST) in the Joint Quantum Institute. The quantum sciences group has done an outstanding
job of bringing in a strong team of a well-established mid-career leader with extensive knowledge, service
laboratory experience, and connections of military systems and needs, and a second, well-established
academic who is a recognized quantum sciences leader. In the past two years under their leadership, an
excellent research team of six experimentalists and one theoretician has been established. In addition, the
group has built impressive research facilities. Among the applications addressed by the quantum sciences
program is absolutely secure communications, a major challenge for a highly mobile and ever-changing
battle scene where your opponent very likely will be able to receive your communications and thus
unbreakable encryption is essential. The ARL team has made a significant effort on different approaches
for a “quantum repeater” to extend the range over which secure communications can be assured.
137

The quantum sciences program is an outstanding example of vision for a future Army need: defining
specific areas that are Army unique, and are not well covered by universities or national laboratories, and
hiring recognized leadership and creating a well-funded, exciting program that has attracted an
outstanding group of early-career scientists and postdoctoral researchers.
Sciences for Lethality and Protection
In the battlefield injury mechanisms area, the management is complimented for having created a
comprehensive program, with talented, energetic scientists who provide the skills needed to define this
area. This is an excellent start, and ARL is to be commended and encouraged to continue to grow this
area.
In the directed energy area, the work on exploiting Raman laser to greatly improve fiber power output
is exceptional and is an archetype for research at ARL that compliments other Department of Defense
(DOD) laboratories while not duplicating academic or industrial research. Another exceptional work is
the nonlinear optical materials and coatings that are frequency agile in the visible spectrum to passively
protect Army optical sensors from directed energy laser threats such as that from straight damage,
jamming, dazzling, and so on.
In the weapons-target interactions area, the advanced penetrator work is a potential game-changer and
is viewed as exceptional.
Information Sciences
Of the reviewed projects, some are deserving of special mention. The research program in electric and
magnetic field sensing is a strong program overall. This is a comprehensive and strongly interconnected
program with projects in sensor development, validation, calibration, algorithm design, and field
deployment. Strong mentorship by senior scientists has effectively grown an impressive cohort of next-
generation scientists to sustain this effort. The private sector partnerships and commercialization activities
are also notable.
Another project related to the detection and characterization of chemical aerosols is worthy of special
attention. This project has successfully demonstrated a laser-based technique for isolating, detecting, and
identifying the chemical compositions of micron-size particles of multiple phases with unprecedented
speed and accuracy. The work is exceptional and novel, and has the ability to revolutionize the aerosol
science field as well as all industries and technologies that rely on aerosol science.
Ongoing work related to the meteorological sensor array (MSA) will enable unprecedented
continuous examination of atmospheric phenomena crucial to our understanding of atmospheric flows
over complex terrain at high horizontal resolution. The unique data that will result from the full
deployment of MSA and its instruments as well as the opportunity to engage multiple partners are factors
that contribute to the high impact of the project.
The planned sensor information test-bed collaborative research environment (SITCORE) facility
appears to be a significant enabler for impactful, multidisciplinary research that expands the scope of
many information sciences projects. The plan to place this facility next to the Network Science Research
Laboratory and to be included as part of the open campus are expected to facilitate collaborative research
and promote innovative solutions to challenge problems.
Computational Sciences
The work presented in the predictive sciences combines machine learning within large simulations to
optimize multiscale model computations with the hierarchical multiscale (HMS) work, with positive
138

results and a promising future. The data-intensive sciences work on neuromorphic processing and
cooperative reinforcement learning was excellent. The panel congratulates the data-intensive sciences
team on its strong start in the new research thrust in machine learning. Alongside operating and managing
high-performance computing (HPC) systems to serve the processing needs of the broader DOD
community, the advanced architecture group has evolved to focus on tactical high-performance
computing at the edge, having made significant progress in evaluating the role of neuromorphic
computing to enable high-fidelity computation using many low-precision elements and very low energy.
Sciences for Maneuver
of information-enriched belief approach (projected functional stochastic gradient-based approach with
teams of robots); and research and simulation work on the Wingman Software Integration Laboratory,
which has a clear path to Army-relevant static and dynamic scenarios and multiple-machine and multiple-
human interactions.
Human Sciences
The real-world behavior (RWB) program now has state-of-the-art expertise in electroencephalogram
(EEG) systems and source localization; it has developed in-house EEG technology and compared it with
commercial EEG systems. Of particular note is a head phantom for EEG, a device approximating the
human skull conduction used to re-create electrical signals on the scalp that will enable the modeling and
exclusion of noise sources, with the goal of identifying measurable neural signals recorded in complex
environments. The group has developed a cutting-edge facility for integrating EEG and other related
neural and physiological sensor data.
The Human Cyber Performance group within the Humans in Multiagent Systems program has
demonstrated foresight into effective collaboration by its initiation of promising work in human aspects of
cybersecurity utilizing the Cyber Human Integrated Modeling and Experimentation Range Army
(CHIMERA) laboratory, jointly developed by ARL’s Human Research and Engineering Computational
and Information Sciences Directorate (CISD).
Analysis and Assessments
The BSVL group has historically led the way in modeling ballistic survivability, lethality, and
vulnerability for the Army, DOD, and international allied community. There is no competition for
leadership in this area—it is ARL’s mission. The movements to embrace HPC to speed computations and
support the Army community needs for analysis and assessment (A&A) are commendable efforts. The
emerging methodology for underbody blast and multihit survivability analyses will be exceptional
contributions to Army A&A.
The finite element modeling of underbody events on vehicles is top-notch work. The finite element
modeling team has a clear understanding of the fidelity required and is advancing state-of-the-art tools
and contributing to their validation.
The approach for the physiological experimental work combining high-speed X-ray imaging with
state-of-the-art and exploratory sensor technologies is an example of outstanding work that is leading this
field. It is the best high-speed X-ray capability that has been observed in this area.
139

In human systems integration (HSI), the human physical accommodation models and soldier
performance and workload modeling and simulation tools developed and employed by ARL are first rate
and have provided the Army and industry with an excellent capability to assess soldier integration into
complex systems. Current tools, including the Improved Performance Research Integration Tool
(IMPRINT) and digital clothing and equipment models, provide analytical capabilities that can be cost-
effectively applied early in the acquisition cycle as well as later during system development.
140

Appendixes


Army Research Laboratory Organization and Science and Technology

Campaign Framework
Figure A.1 is an organization chart for the Army Research Laboratory (ARL), Figure A.2 is ARL’s
science and technology (S&T) campaign framework, and Table A.1 maps the ARL organizational chart to
the campaign areas reviewed in 2017.
Office of the Director
Associate Director for

Laboratory Operations Director’s Staff
Army Research Office
Vehicle Human Research Survivability/ Computational and Sensors and Weapons and
Technology and Engineering Lethality Analysis Information Sciences Electron Devices Materials Research
Directorate Directorate Directorate Directorate Directorate Directorate
FIGURE A.1 Army Research Laboratory organization chart.
143

FIGURE A.2 The Army Research Laboratory (ARL) science and technology (S&T) campaign
framework includes Extramural Basic Research, Computational Sciences, Materials Research, and
Analysis and Assessment. These four crosscutting campaigns act in concert with ARL’s four focused
campaigns, Human Sciences, Information Sciences, Sciences for Lethality and Protection, and Sciences
for Maneuver, leading to knowledge products and technologies.
144

TABLE A.1 Mapping of the Army Research Laboratory (ARL) Organization Chart to the Science and
Technology Campaign Areas Reviewed in 2017
Campaign Topic ARL Directorate Involved
Materials Research Energy-efficient electronics and photonics SEDD, WMRD
Materials for soldier and platform power
systems
Quantum sciences
Sciences for Lethality and Battlefield injury mechanisms WMRD, SLAD, SEDD, CISD
Protection Directed energy
Weapon-target interactions
Information Sciences Sensing and effecting CISD, SEDD, HRED

System intelligence and intelligent systems
Human and information interaction
Atmospheric sciences
Computational Sciences Advanced computing architectures CISD, SLAD, SEDD, VTD,

Data-intensive sciences WMRD
Predictive sciences
Sciences for Maneuver Intelligence and control VTD, HRED, SEDD, CISD
Machine-human interaction
Perception
Human Sciences Humans in cybersecurity HRED, SLAD, CISD

Humans in multiagent systems
Human variability
Real-world behavior
Analysis and Assessment Ballistics survivability, lethality, and SLAD, HRED

vulnerability
Personnel survivability
Human systems integration
NOTE: CISD, Computational and Information Sciences Directorate; HRED, Human Research and Engineering
Directorate; SEDD, Sensors and Electron Devices Directorate; SLAD, Survivability and Lethality Analysis
Directorate; VTD, Vehicle Technology Directorate; and WMRD, Weapons and Materials Research Directorate.
145

Army Research Laboratory Technical Assessment Board Members and Staff

Biographical Information
JENNIE S. HWANG, Chair, is CEO of H-Technologies Group, and board trustee and distinguished
adjunct professor at Case Western Reserve University. Her career encompasses corporate and
entrepreneurial businesses, international collaboration, research management, technology transfer and
global leadership positions, as well as corporate and university governance. Among her many honors and
awards are U.S. Congressional Certificates of Recognition; induction into the International Hall of
Fame—Women in Technology and Ohio Women Hall of Fame; named in the R&D-Stars-to-Watch;
distinguished alumni awards; honorary doctoral degree; and YWCA Achievement Award. She was the
CEO of International Electronic Materials and has held senior executive positions with Lockheed Martin,
Hanson PLC, and Sherwin-Williams and co-founded entrepreneurial businesses. She is internationally
recognized as a pioneer and long-standing leader in the infrastructure development of electronics
miniaturization and green manufacturing. She has served as global president of the Surface Mount
Technology Association and in other global leadership positions. An international speaker and author of
more than 475 publications, including several internationally available books, she has lectured to tens of
thousands of managers, engineers, and researchers on professional development courses. Her speeches
range from university commencement addresses to the keynote at the Department of Defense (DOD)
Federal Women’s Program to tutorials at the U.S. Patent and Trademark Office. She is also a prolific
author and speaker on education, workforce, and social and business issues. Additionally, Dr. Hwang has
served as a board director for Fortune 500 NYSE-traded and private companies and various university
and civic boards, and on the International Advisory Board of the Singapore Advanced Technology and
Manufacturing Institute and a number of international industry boards. She is a member of the National
Academy of Engineering (NAE) and chairs the Technical Assessment Board of the Army Research
Laboratory (ARL) of the National Academies of Sciences, Engineering, and Medicine and has served as
NAE Membership Search Executive (Materials Section) and on the National Materials and Manufacturing
Board, DOD R&D Globalization Board, Committee on Forecasting Future Disruptive Technologies, and
NAE Award Committee, among others. She also has served as a reviewer for National Academies reports
and other national/international publications. Her formal education includes the Harvard University
Executive Program, Columbia University Business School Governance Program, and four academic
degrees (Ph.D., M.A., M.S., B.S.) in materials science and metallurgical engineering, chemistry, and
liquid crystal science. The Dr. Jennie S. Hwang Award for Faculty Excellence was established at her alma
maters. The Dr. Jennie S. Hwang YWCA Award, established in her honor and now running for 17 years,
encourages and recognizes outstanding women students in STEM.
MARK EBERHART is a professor in the Department of Chemistry and Geochemistry at the Colorado
School of Mines, where he directs the Molecular Theory Group (MTG). At the MTG, knowledge of
bonding is obtained through detailed topological analyses of the spatial distribution of electrons in
molecules and solids. Many subtle aspects of the distribution become obvious when viewed from a
topological perspective. The accompanying topological formalism gives well-defined, unambiguous,
meaningful, and consistent definitions to previously indeterminate quantities such as atomic bonds and
146

basins. His work is based primarily on first principles computations, which provide the electron charge
densities, and topological analysis software developed at the MTG. He is also exploring the topological
and geometric origins responsible for the stability of amorphous metallic alloys. In addition to his work
on condensed phase systems, his group has active research programs exploring the relationships between
charge density and the chemical properties of molecular systems, both organic and inorganic. Dr.
Eberhart holds a B.S. degree in chemistry and applied mathematics from the University of Colorado, an
M.S. degree in physical miochemistry from the University of Colorado, and a Ph.D. in materials science
and engineering from the Massachusetts Institute of Technology (MIT).
GEORGE (RUSTY) T. GRAY III is a laboratory fellow and staff member in the Dynamic Properties and
Constitutive Modeling Team within the Materials Science Division of Los Alamos National Laboratory
(LANL). He came to LANL following a 3-year visiting scholar position at the Technical University of
Hamburg-Harburg in Hamburg, Germany, having received his Ph.D. in materials science in 1981 from
Carnegie Mellon University. As a staff member (1985-1987) and later team leader (1987-2003) in the
Dynamic Materials Properties and Constitutive Modeling Team within the Structure/Property Relations
Group (MST-8) at LANL, he has directed a research team working on investigations of the dynamic
response of materials. He conducts fundamental, applied, and focused programmatic research on
materials and structures, in particular in response to high strain rate and shock deformation. His research
is focused on experimental and modeling studies of substructure evolution and mechanical response of
materials. These constitutive and damage models are utilized in engineering computer codes to support
large-scale finite element modeling simulations of structures ranging from national defense (the
Department of Energy, DOD, the Defense Advanced Research Projects Agency), industry (GM, Ford,
Chrysler, and Bettis), foreign object damage, and manufacturing. He is a life member of Clare Hall,
University of Cambridge, U.K., where he was on sabbatical in the summer of 1998. He co-chaired the
Physical Metallurgy Gordon Conference in 2000. He is a fellow of the American Physical Society (APS),
a fellow of ASM International (ASM), and a fellow of the Minerals, Metals, and Materials Society
(TMS). He is a member of APS, ASM, TMS, and serves on the International Scientific Advisory Board of
the European DYMAT Association. In 2010, he served as the president of TMS. Starting in 2012, he
became the chair of the Acta Materialia board of governors, which oversees the publication of the
journals Acta Materialia, Scripta Materialia, and Acta Biomaterialia. He has authored or co-authored
over 430 technical publications. In 2017, he was elected to the NAE.
PRABHAT HAJELA is provost and professor of mechanical and aerospace engineering at the Rensselaer
Polytechnic Institute. His research interests include analysis and design optimization of multidisciplinary
systems, system reliability, emergent computing paradigms for design, artificial intelligence, and machine
learning in multidisciplinary analysis and design. Before joining Rensselaer, he worked as a research
fellow at the University of California, Los Angeles, for a year and was on the faculty at the University of
Florida for 7 years. He has conducted research at NASA’s Langley and Glenn Research Centers and the
Eglin Air Force Armament Laboratory. In 2003, Hajela served as a congressional fellow responsible for
science and technology policy in the Office of U.S. Senator Conrad Burns (R-Mont.). He worked on
several legislative issues related to aerospace and telecommunications policy, including the anti-SPAM
legislation that was signed into law in December 2003. Hajela is a fellow of the American Institute of
Aeronautics and Astronautics (AIAA), a fellow of the Aeronautical Society of India (AeSI), and a fellow
of the American Society of Mechanical Engineers (ASME). Hajela has held many editorial assignments,
including editor of Evolutionary Optimization and associate editor of the AIAA journal, and is on the
editorial board of six other international journals. He has published over 270 papers and articles in the
areas of structural and multidisciplinary optimization and is an author or co-author of four books in these
areas. In 2004, he was the recipient of AIAA’s Biennial Multidisciplinary Design Optimization Award.
WESLEY L. HARRIS is the Charles Stark Draper Professor of Aeronautics and Astronautics and director
of the Lean Sustainment Initiative at MIT. He was elected to the NAE “for contributions to understanding
147

of helicopter rotor noise, for encouragement of minorities in engineering, and for service to the
aeronautical industry.” He has performed research and published in refereed journals in the following
areas: fluid mechanics; aerodynamics; unsteady, nonlinear aerodynamics; acoustics; lean manufacturing
processes; and military logistics and sustainment. Mr. Harris has substantial experience as a leader in
higher education administration and management. He also has demonstrated outstanding leadership in
managing major national and international aeronautical and aviation programs and personnel in the
executive branch of the federal government. Mr. Harris is an elected fellow of the AIAA, AHS, and NTA
for personal engineering achievements, engineering education, management, and advancing cultural
diversity.
WILLIAM S. MARRAS is the Honda Chair Professor in the Department of Integrated Systems
Engineering at Ohio State University, and holds joint appointments in the Departments of Orthopaedic
Surgery, Physical Medicine, and Neurosurgery. Dr. Marras is also executive director and scientific
director of the Spine Research Institute and the executive director of the Institute for Ergonomics. His
research is centered on understanding the role of biomechanics in spine disorder causation and its role in
the prevention, evaluation, and treatment of spine disorders. His research includes epidemiologic studies,
laboratory biomechanics studies, mathematical modeling, and clinical studies. His findings have been
published in over 200 peer-reviewed journal articles, and have been cited over 15,000 times. He also has
written numerous books and book chapters, including his most recent book, titled The Working Back: A
Systems View. He is a member of the NAE and holds fellow status in six professional societies including
the American Society for the Advancement of Science (AAAS) and has been widely recognized for his
contributions through numerous national and international awards including two Volvo Awards for low
back pain research. Professor Marras has been active in the National Academies, having served on over a
dozen boards and committees, and has served as chair of the Board on Human Systems Integration for
multiple terms. He has also served as editor-in-chief of Human Factors, and is currently deputy editor of
Spine and is the immediate past president of the Human Factors and Ergonomics Society. Dr. Marras
recorded a TEDx talk titled “Back Pain and Your Brain” and was recently featured on NPR’s All Things
Considered. He received a B.S. in engineering from Wright State University, an M.S.in industrial
engineering from Wayne State University, a Ph.D. in bioengineering and ergonomics from Wayne State
University, and a D.Sc. Honoris Causa from the University of Waterloo.
ALAN NEEDLEMAN is University Distinguished Professor and TEES Distinguished Research

Professor in the Department of Materials Science and Engineering at Texas A&M University. Topics of
particular interest have been the micromechanics of ductile fracture by the nucleation, growth, and
coalescence of microvoids; brittle-ductile transitions; material and structural instabilities; relations
between microstructure and mechanical properties in heterogeneous solids; and dynamic crack growth.
He is a fellow of the American Society of Mechanical Engineers, and the American Academy of
Mechanics, and a member of NAE, the American Academy of Arts and Sciences, and the Academy of
Medicine, Engineering and Science of Texas.
DANIEL A. REED is the vice president for research and economic development at the University of Iowa
(named in September 2012). He is also the University Computational Science and Bioinformatics chair,
and professor of computer science and electrical and computer engineering. He was corporate vice
president at Microsoft from 2009 to 2012, responsible for global technology policy and extreme
computing, and director of scalable and multicore computing at Microsoft from 2007 to 2009. He founded
the Renaissance Computing Institute in 2004 and served as its director until December 2007. He was also
Chancellor’s Eminent Professor and served as senior adviser for strategy and innovation to Chancellor
James Moeser, University of North Carolina (UNC), Chapel Hill. He served as CIO and vice chancellor
for information technology services at UNC, Chapel Hill, from June 2004 through April 2007. Prior to
that, he was director of the National Center for Supercomputing Applications (NCSA), Gutgsell Professor
and head of the Department of Computer Science at the University of Illinois, Urbana-Champaign. He
148

was appointed to the President’s Council of Advisors on Science and Technology (PCAST) by President
George W. Bush in 2006, and served on the President’s Information Technology Advisory Committee
(PITAC) from 2003 to 2005. As chair of PITAC’s computational science subcommittee, he was lead
author of the report Computational Science: Ensuring America’s Competitiveness. On PCAST, he co-
chaired the Networking and Information Technology subcommittee (with George Scalise of the
Semiconductor Industry Association) and co-authored a report on the Networking and Information
Technology Research and Development (NITRD) program called Leadership Under Challenge:
Information Technology R&D in Competitive World. He is also a member of PCAST’s Personalized
Medicine subcommittee. Dr. Reed is the past chair of the board of directors of the Computing Research
Association (CRA) and currently serves on its Government Affairs Committee. CRA represents the
research interests of the university, national laboratory, and industrial research laboratory communities in
computing across North America. Dr. Reed received his B.S. (summa cum laude) in computer science
from the University of Missouri, Rolla, in 1978, and his M.S. and Ph.D. in computer science from Purdue
University in 1980 and 1983.
Staff
AZEB GETACHEW is a senior program assistant at the Laboratory Assessments Board (LAB). She
joined the LAB in March 2017 and is responsible for administrative and logistical planning for project
meetings and other activities. She previously worked as an interim administrative assistant in several
administrative capacities at the National Academies including the LAB, the Naval Studies Board, and the
Institute of Medicine. Ms. Getachew has an associate of applied science degree in information systems
from Columbia Union College, which is now Washington Adventist University.
EVA LABRE is the administrative coordinator for the LAB. Since 2009, she has been responsible for
assisting in the management of the administrative aspects of panel formation, panel meetings, report
publication and dissemination, and program development. In addition, she has been responsible for travel
expense accounting. In 2014, she was promoted and has recently taken on more responsibilities related to
financial aspects of the work of the LAB. Ms. Labre previously held administrative positions at the
National Academies on the staff of the Committee on International Organizations and Programs in the
Office of International Affairs and on the staff of the Research Associateship Program in the Office of
Scientific and Engineering Personnel. Ms. Labre has a B.A. in art history from George Washington
University.
JAMES P. MCGEE is the director of the LAB, the Army Research Laboratory Technical Assessment
Board (ARLTAB), and the Committee on the National Institute of Standards and Technology Technical
Programs, all within DEPS at the National Academies. Since 1994, he has been a senior staff officer at
the National Academies, directing projects in the areas of systems engineering and applied psychology,
including activities of ARLTAB and projects of the Committee on National Statistics’ Panel on
Operational Testing and Evaluation of the Stryker Vehicle and the Committee on Assessing the National
Science Foundation’s Scientists and Engineers Statistical Data System, the Committee on the Health and
Safety Needs of Older Workers, and the Steering Committee on Differential Susceptibility of Older
Persons to Environmental Hazards. He has also served as staff officer for the National Academies’
projects on air traffic control automation, musculoskeletal disorders and the workplace, and the changing
nature of work. Prior to joining the National Academies, Dr. McGee held technical and management
positions in systems engineering and applied psychology at IBM, General Electric, RCA, General
Dynamics, and United Technologies. He received his B.A. from Princeton University and his Ph.D. from
Fordham University, both in psychology, and for several years instructed postsecondary courses in
applied psychology and in organizational management.
149

ARUL MOZHI is senior program officer at the LAB. Since 1999, he has been directing projects in the
areas of defense science and technology, including those carried out by numerous study committees of the
LAB, the ARLTAB, the Naval Studies Board, and the National Materials and Manufacturing Board. Prior
to joining the National Academies, Dr. Mozhi held technical and management positions in systems
engineering and applied materials research and development at UTRON, Roy F. Weston, and Marko
Materials. He received his M.S. and Ph.D. degrees (the latter in 1986) in materials engineering from the
Ohio State University and then served as a postdoctoral research associate there. He received his B.Tech.
in metallurgical engineering from the Indian Institute of Technology, Kanpur, in 1982.
150

Assessment Criteria
The Army Research Laboratory Technical Assessment Board (ARLTAB) assessment considered the
following general questions posed by the ARL director:
 Is the scientific quality of the research of comparable technical quality to that executed in
leading federal, university, and industrial laboratories both nationally and internationally?
 Does the research program reflect a broad understanding of the underlying science and
research conducted elsewhere?
 Does the research employ the appropriate laboratory equipment and numerical models?
 Are the qualifications or the research team compatible with the research challenge?
 Are the facilities and laboratory equipment state of the art?
 Are programs crafted to employ the appropriate mix of theory, computation, and
experimentation?
To assist ARL in addressing promising technical approaches, the Board will also consider the
following questions:
 Are there especially promising projects that, with improved direction or resources, could
produce outstanding results that can be transitioned ultimately to the field?
 Are there promising outside-the-box concepts that should be pursued but are not currently
in the ARL portfolio?
The ARLTAB applied the following metrics or criteria to the assessment of the scientific and
technical work reviewed at the ARL:
Project Goals and Plans
 Are the objectives clearly stated and are tasks well defined to achieve objectives?
 Are milestones defined? Are they appropriate? Do they appear feasible?
 Are obstacles and challenges defined (technical, resources, time)?
 Assuming success, what difference will it make to the science base, to the end user, or in
a mission area context?
 Does the project plan identify dependencies (i.e., successes depend on success of other
activities within the project or on the success of projects developed outside ARL)?
 Does the project represent an area where application of ARL strengths is appropriate?
 What stopping rules, if any, are being or should be applied?
151

Methodology and Approach
 Are the methods (e.g., laboratory experiment, modeling/simulation, field test, analysis)
appropriate to the problems? Do these methods integrate?
 Are the hypotheses appropriately framed within the literature and theoretical context?
 Is there an alternative approach that facilitates the progress of the project?
 Is there a clearly identified and appropriate process for performing required analyses,
prototypes, models, simulations, tests, and so on?
 Is the data collection and analysis methodology appropriate?
 Are conclusions supported by the results?
 Are proposed ideas for further study reasonable?
 Do the trade-offs between risk and potential gain appear reasonable?
 If the project demands technological or technical innovation, is that occurring?
Capabilities and Resources
 If staff or equipment is not adequate, how might the project be triaged (which technical
thrust should be emphasized, which sacrificed?) to best move toward its stated
objectives?
 Will the project recruit new talent into ARL?
Scientific Community
 Presentations and colloquia.

 Participation in professional activities (society officers, conference committees, journal
editors).
 Papers in quality refereed journals and conference proceedings (and their citation index).
 Educational outreach (serving on graduate committees, teaching/lecturing, invited talks,
mentoring students).
 Fellowships and awards (external and internal).
 Participation on review panels (Army Research Office, National Science Foundation,
Multidisciplinary University Research Initiative, etc.).
 Patents and intellectual property and examples of how the patent or intellectual property
is used.
 Involvement in building an ARL-wide cross-directorate community.
 Public recognition, for example, in the press and elsewhere, for ARL research.
 Collaborations (lead, partner, support).
152

Acronyms
2D two dimensional
3D three dimensional
4D four dimensional
A&A analysis and assessment

ABLE atmospheric boundary layer environment
ABM agent-based model
AEM alkaline exchange membrane
AFRL Air Force Research Laboratory
AI artificial intelligence
AIS Abbreviated Injury Scale
AM additive manufacturing
AMT Amazon Mechanical Turk
ANL Argonne National Laboratory
API application programming interface
ARDEC Armament Research, Development and Engineering Center
ARL Army Research Laboratory
ARLTAB Army Research Laboratory Technical Assessment Board
ARO Army Research Office
ATGM anti-tank guided missile
BABT behind-armor blunt trauma

BED Battlefield Environments Division
BSVL ballistics survivability, vulnerability, and lethality
C4ISR command, control, communications, computers, intelligence, surveillance, and

reconnaissance
CCE core campaign enabler
CHIMERA Cyber Human Integrated Modeling and Experimentation Range Army
CISD Computational and Information Sciences Directorate
CMOS complementary metal-oxide-semiconductor
COTS commercial off-the-shelf
CRA Collaborative Research Alliance
CRADA cooperative research and development agreement
CREB Center for Research in Extreme Batteries
CSM Center for Semiconductor Modeling
CT computed tomography
CTA Collaborative Technology Alliance
CW continuous wave
153

DAHI Diverse Accessible Heterogeneous Integration

DARPA Defense Advanced Research Projects Agency
DE directed energy
DFT density functional theory
DIC digital image correlation
DOD Department of Defense
DoE design of experiments
DOE Department of Energy
EEG electroencephalogram
ERA essential research area
ET eye tracking
EW electronic warfare
FCC face-centered cubic

FEM finite element method
FP floating point
FPGA field programmable gate array
FRM fast-running model
FY fiscal year
GDTA goal-directed task analysis

GPS global positioning system
GPU graphics processing unit
HAT human-agent teaming

HCI human-computer interaction
HDR high dynamic range
HER hydrogen evolution reaction
HFE human factors engineering
HIDRA high-voltage in situ diagnostic radiography apparatus
HII human-information interaction
HMI human-machine interaction
HMS hierarchical multiscale
HPC high-performance computing
HRED Human Research and Engineering Directorate
HRI human-robot interaction
HSI human systems integration
ICRA International Conference on Robotics and Automation

IED improvised explosive device
IEEE Institute of Electrical and Electronics Engineers
IMPRINT Improved Performance Research Integration Tool
I/O input/output
IoBT Internet of Battlefield Things
IP Internet protocol
IR infrared
ISC Intelligent Systems Center
IT information technology
154

JASS Job Assessment Software System

JER Jornada Experimental Range
JP-8 jet propellant 8 (fuel)
JTAPIC Joint Trauma Analysis and Prevention of Injury in Combat
KCI Key Campaign Initiative
LAB Laboratory Assessments Board

LBM lattice Boltzmann method
LIDAR light detection and ranging
MATERHORN mountain terrain atmospheric modeling and observations

MBE molecular beam epitaxy
MD molecular dynamics
MEMS microelectromechanical system
MHI machine-human interaction
ML machine learning
MODE method for object-based diagnostic evaluation
MOS military occupational specialty
MOSFET metal-oxide semiconductor field-effect transistor
MPI message passing interface
MRMC Medical Research and Materiel Command
MS&PP materials for soldier and platform power systems
MSA meteorological sensor array
MURI Multidisciplinary University Research Initiative
MUVES modular UNIX-based vulnerability estimation suite
NASA National Aeronautics and Space Administration

NCAR National Center for Atmospheric Research
NIST National Institute of Standards and Technology
NoC Network-on-Chip
NRL Naval Research Laboratory
NSA National Security Agency
NSF National Science Foundation
NWP numerical weather prediction
OPBO output power back off

ORCA operational-requirements-based casualty assessment
PA power amplifier
PAE power amplifier high-energy efficiency
PBL planetary boundary layer
PDR preliminary design review
PDV photon Doppler velocimetry
PEM proton-exchange membrane
PI principal investigator
PM program manager
POMDP Partially Observable Markov Decision Process
PPE personal protective equipment
PS personnel survivability
PZT lead zirconate titanate
155

R&D research and development

RF radio frequency
RFI radio frequency interference
RWB real-world behavior
S&E sensing and effecting

S&T science and technology
SA situation awareness
SBIR Small Business Innovation Research
SEDD Sensors and Electron Devices Directorate
SEM scanning electron microscopy
SHPB split-Hopkinson pressure bar
SIIS system intelligence and intelligent systems
SLAD Survivability and Lethality Analysis Directorate
SMA shape memory alloy
SME subject matter expert
SRNC Semiconductor Research Nanofabrication Center
SWAP size, weight, and power
SWAPC size, weight, power, and cost
SWAPCT size, weight, power, cost, and time
SWAPT size, weight, power, and time
SWAPTN SWAPT plus network
TAD target audience description

TARDEC Tank Automotive Research, Development, and Engineering Center
TBI traumatic brain injury
ToA time of arrival
TPU tensor processing unit
TRL technology readiness level
TTCP The Technical Cooperation Program
TUEI tactical unit energy independence
UAS unmanned aircraft system

UAV unmanned aerial vehicle
UHMWPE ultrahigh molecular weight polyethylene
UQ uncertainty quantification
USDA U.S. Department of Agriculture
UV ultraviolet
V&V validation and verification

VFM vortex filament method
VI vehicle intelligence
VLES very-large-eddy-simulation
VoI value of information
VTD Vehicle Technology Directorate
WBG wide bandgap

WMRD Weapons and Materials Research Directorate
WRF weather research and forecasting
WSMR White Sands Missile Range
156

2017-2018 Assessment of The Army Research Laboratory PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2017-2018 Assessment of The Army Research Laboratory PDF

Uploaded by

Copyright:

Available Formats

THE NATIONAL ACADEMIES PRESS

This PDF is available at http://nap.edu/25011 SHARE

2017-2018 Assessment of the Army Research Laboratory:

168 pages | 8.5 x 11 | PAPERBACK

– Access to free PDF downloads of thousands of scientiﬁc reports

Copyright © National Academy of Sciences. All rights reserved.

Army Research Laboratory Technical Assessment Board

Laboratory Assessments Board

Division on Engineering and Physical Sciences

A Consensus Study Report of

Copyright National Academy of Sciences. All rights reserved.

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001

International Standard Book Number-13: 978-0-309-47161-9

Copyright 2018 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Proceedings published by the National Academies of Sciences, Engineering, and

Copyright National Academy of Sciences. All rights reserved.

ARMY RESEARCH LABORATORY TECHNICAL ASSESSMENT BOARD

JENNIE S. HWANG, NAE,1 H-Technologies Group, Chair

AZEB GETACHEW, Senior Program Assistant

PANEL ON ASSESSMENT AND ANALYSIS

ALAN NEEDLEMAN, NAE, Texas A&M University, Chair

PANEL ON BALLISTICS SCIENCE AND ENGINEERING

Copyright National Academy of Sciences. All rights reserved.

ALLAN T. MENSE, Raytheon Missile Systems

PANEL ON COMPUTATIONAL SCIENCES

DANIEL A. REED, University of Iowa, Chair

PANEL ON HUMAN FACTORS SCIENCE

WILLIAM S. MARRAS, NAE, Ohio State University, Chair

Copyright National Academy of Sciences. All rights reserved.

PANEL ON INFORMATION SCIENCE

PRABHAT HAJELA, Rensselaer Polytechnic Institute, Chair

PANEL ON MATERIALS SCIENCE AND ENGINEERING

MARK E. EBERHART, Colorado School of Mines, Chair

PANEL ON MECHANICAL SCIENCE AND ENGINEERING

WESLEY L. HARRIS, NAE, Massachusetts Institute of Technology, Chair

Copyright National Academy of Sciences. All rights reserved.

LABORATORY ASSESSMENTS BOARD

ROSS B. COROTIS, NAE, University of Colorado, Boulder, Chair

AZEB GETACHEW, Senior Program Assistant

Copyright National Academy of Sciences. All rights reserved.

Amy E. Duwel, The Charles Stark Draper Laboratory, Inc.,

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

3 SCIENCES FOR LETHALITY AND PROTECTION 38

6 SCIENCES FOR MANEUVER 89

Copyright National Academy of Sciences. All rights reserved.

Conclusions and Recommendations, 102

7 HUMAN SCIENCES 105

8 ANALYSIS AND ASSESSMENT 123

9 CROSSCUTTING CONCLUSIONS AND RECOMMENDATIONS AND EXCEPTIONAL

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Energy-Efficient Electronics and Photonics

Materials for Soldier and Platform Power Systems

Copyright National Academy of Sciences. All rights reserved.

SCIENCES FOR LETHALITY AND PROTECTION

Battlefield Injury Mechanisms

Copyright National Academy of Sciences. All rights reserved.

Penetration, Armor, and Adaptive Protection

Sensing and Effecting