Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Office of Research & Development

print icon sign up for VA Research updates

Strategic Partnerships

Why Strategic Partnerships in AI?

Collaboration with other researchers and AI specialists is vital for the NAII to pioneer new AI R&D and applications, drive improvement in operations, and advance scientific scholarship – all to provide the best possible outcomes for our veterans.

Here are some examples of strategic partnerships for research and development (R&D):

  1. Physical health and well-being. What explains differences in well-being, and what are the best ways AI can improve it?
  2. Veteran workforce. How have Veterans fared in the labor market, and what are the best ways to help them transition from service to the civilian sector and progress throughout their career, including via AI?
  3. Ethics and public policy. What principles should guide the development and application of AI, and how should they be introduced in policy?

Are you a researcher with an idea and interested in working with the NAII? Contact us at naii@va.gov to pitch your idea and see where there is an opportunity to collaborate.

Sample Strategic Partnerships

The NAII is proud of our strategic partnerships that are contributing to the scientific body of knowledge and overall understanding and application of AI. Here is a sampling of collaborations with our strategic partners, grouped by area of focus.


Physical Health and Well-being

The NAII uses AI and machine learning to help identify the structural factors that contribute to differences in physical health and well-being and to provide Veterans with recommendations on behaviors that may improve their overall well-being.

The Impact of Socio-economic Factors on Veteran Health and Well-being

What are the primary contributing factors towards differences in physical and overall well-being among Veterans? The NAII uses machine learning to break down the role that demographic (e.g., age), geographic (e.g., rurality), and socio-economic (e.g., financial anxiety) factors play among Veterans' stated physical and overall well-being. The results show that, while demographic and geographic factors matter, socio-economic characteristics matter overwhelmingly more. The results also suggest that solutions to mental and physical health challenges among Veterans must be holistic and address their workplace experience and livelihood.

Leveraging Machine Learning to Characterize the Role of Socio-economic Determinants on Physical Health and Well-being Among Veterans
Computers in Biology and Medicine
Makridis, Christos A., David Zhao, Adi Bejan, and Gil Alterovitz. (2021)

Understanding the contribution of demographic, socio-economic, and geographic characteristics as determinants of physical health and well-being is essential for guiding public health policies and preventative behavior interventions, particularly with the spread of coronavirus. We leverage several machine learning methods to build predictive models of overall well-being and physical health among veterans as a function of these three sets of characteristics. We link Gallup's US Daily Poll between 2014 and 2017 over a range of demographic and socio-economic characteristics with zip code characteristics from the Census Bureau to build predictive models of overall and physical well-being. Although the predictive models of overall well-being have weak performance, our classification of low levels of physical well-being performed better. Gradient boosting delivered the best results (80.2% precision, 82.4% recall, and 80.4% AUROC) with perceptions of purpose in the workplace and financial anxiety as the most predictive features. Our results suggest that additional measures of socio-economic characteristics are required to better predict physical well-being, particularly among vulnerable groups, like veterans. Reliable and effective predictive models will provide opportunities to create real-time and personalized feedback to help individuals improve their quality of life.

Using AI and Machine Learning to Identify COVID-19 Risk Factors in Veterans

How can AI applications be designed so that they complement clinician's work, rather than become substitutes for them in the healthcare sector? Drawing upon the urgency of the COVID-19 pandemic as an opportunity to serve patients more effectively, machine learning is used to create a risk factor for each Veteran that enters a local VA medical center (VAMC) as a function of their medical history, labs, and vitals. The NAII subsequently developed an accessible user interface for clinical applications. The interface allows clinicians to see each patient's risk factor, how it changes, and the risk score's primary contributing factors. While the interface does not tell clinicians how to treat their patients, it empowers them with relevant information that would otherwise be too complicated to process at scale.

Designing Covid-19 Mortality Predictions to Advance Clinical Outcomes: Evidence from the Department of Veterans Affairs.
Accepted, BMJ Health & Care Informatics
Makridis, Christos A., Tim Strebel, Vince Marconi, and Gil Alterovitz. (2021)

Using administrative data on all veterans who enter the Department of Veterans Affairs (VA) medical centers throughout the United States, this paper uses machine learning methods to predict mortality rates for COVID-19 patients between March and August 2020. First, using comprehensive data on over 10,000 veterans' medical history, demographics, and lab results, we estimate five AI models. Our XGBoost model performs the best, producing an AUROC and AUPRC of 0.87 and 0.41, respectively. Second, through a unique collaboration with the Washington DC VA medical center, we develop a dashboard that incorporates these risk factors and the contributing sources of risk, which we deploy across local VA medical centers.

Predicting COVID-19 Outcomes for Veterans Using AI

Researchers typically do not have access to highly granular data on individuals or the locations that they live. Using data on 122 Veteran Healthcare Systems, the NAII explored how socio-demographic characteristics help predict healthcare outcomes among Veterans. The study found that even the inclusion of information about the county that an individual lives in can help predict COVID-19 infection and mortality outcomes.

How Much Does the (Social) Environment Matter? Using Artificial Intelligence to Predict COVID-19 Outcomes with Socio-demographic Data
Pacific Symposium on Biocomputing
Makridis, Christos A., Anish Mudibe and Gil Alterovitz. (2020)

While the coronavirus pandemic has affected all demographic brackets and geographies, certain areas have been more adversely affected than others. This paper focuses on Veterans as a potentially vulnerable group that might be systematically more exposed to infection than others because of their co-morbidities, i.e., greater incidence of physical and mental health challenges. Using data on 122 Veteran Healthcare Systems (HCS), this paper tests three machine learning models for predictive analysis. The combined LASSO and ridge regression with five-fold cross-validation performs the best. We find that socio-demographic features are highly predictive of both cases and deaths–even more critical than any hospital-specific characteristics. These results suggest that socio-demographic and social capital characteristics are important determinants of public health outcomes, especially for vulnerable groups, like Veterans, and should be investigated further.




Veteran Workforce

The NAII is using AI to understand the disparities in Veterans' labor market experiences and their transitions across jobs. This information is being utilized so that matching services can be improved, and benefits can be used towards the educational investments that are likely to have the greatest payoffs.

Why is There a Pay Gap Between Public and Private Sectors?

The federal government faces a severe shortage of skilled workers, especially in technical roles that require AI skills. Using information on thousands of high skilled workers observed for over a decade throughout the public and private sectors, the NAII quantifies the role that financial versus non-pecuniary considerations play in influencing the decision to join or leave the federal government. The results suggest that financial factors, while present, are not as important as the non-pecuniary considerations, especially those relating to the presence of career and advancement opportunities.

(Why) Is There a Public/Private Pay Gap?
Accepted, Journal of Government and Economics.
Makridis, Christos A. (2021).

The government is facing a severe shortage of skilled workers. The conventional wisdom in branches of policy and public administration is that this worker deficit is driven by uncompetitive salaries that are not attracting top talent. Using longitudinal data on high skilled workers between 1993 and 2013, this paper shows that, if anything, government employees earn more than their private-sector counterparts. Although government workers tend to earn less in the raw data, these differences are driven by the correlation between unobserved productivity and selection into private-sector jobs. Instead, this paper provides empirical evidence that low non-pecuniary amenities, such as development opportunities and management, can explain earnings differences between the public and private sectors.

What Challenges Do Veterans Experience in the Labor Market?

Given that Veterans' experience in the workplace is an important contributing factor to their mental and physical health, understanding the labor market outcomes among Veterans relative to non-Veterans is a priority. The NAII analyzed data on millions of workers between 2005 and 2018 and found that Veterans earn slightly less than their counterparts, even within similar industries and occupations. The difference in earnings is exacerbated by the increased clustering of Veterans within cities with weaker labor market prospects and Veteran educational investments that tend to generate lower returns than their counterparts. The results highlight that, while there are some the heightened challenges that Veterans face in the labor market, there are also considerable opportunities for growth among Veterans who earn a science, technology, engineering, or math credential.

How AI Jobs Impact Economic Growth

Can AI impact well-being and the structure of the economy? Using data across 343 cities in the United States, the NAII quantified the relationship between the expansion of the AI job market and both subjective well-being and economic activity. It was found that the expansion of the AI job market has positively impacted well-being primarily through the improvements in the local economy. The results highlight the importance of modernizing the services sector so that workers can benefit from AI advances, whether in their personal or work lives.

(How) Does Artificial Intelligence Raise Well-being? The Service Revolution and Economic Growth
Invited for special issue, Journal of Services Research
Makridis, Christos A., and Saurabh Mishra. (2021)

The share of artificial intelligence (AI) jobs in total job postings have increased from 0.2% to nearly 1% between 2010 to 2019, but there is significant heterogeneity across cities in the United States (US). Using new data on the AI job postings across 343 US cities, combined with data on subjective well-being and economic activity, we uncover the central role that service-based cities play to translate the benefits of AI job growth to subjective well-being. We find that cities with higher growth in AI job postings witnessed higher economic growth. The relationship between AI job growth and economic growth is driven by cities with higher concentrations of modern (or professional) services. AI job growth also leads to an increase in the state of well-being. The transmission channel of AI job growth to increased subjective well-being is explained by the positive relationship between AI jobs and economic development. These results are consistent with structural transformation models where technological change leads to improvements in well-being through improvements in economic activity. By empowering the modern service economy across cities, our results suggest that AI-driven economic growth could also raise overall well-being and social welfare.




Ethics and Public Policy

The NAII is examining the underlying principles that must be embedded into AI R&D so that every AI application ultimately benefits the end user–Veterans and society.

How Can AI Applications Benefit Veterans?

How can it be ensured that AI applications benefit, rather than harm, patients, especially vulnerable groups, like Veterans? Drawing upon the recently developed principles around the trustworthy use of AI, the NAII applied these principles from the perspective of Veterans and emphasized how AI applications should be prioritized when: (a) the benefits of use significantly outweigh the risks (and the risks are assessed and managed), (b) the application is confined to a well-defined domain with clear purposes, and (c) the operations and outcomes are interpretable and accessible.

AI Ethics and Veterans' Perspective

There is widespread agreement that, while artificial intelligence offers significant potential benefits for individuals and society at large, there are also serious challenges to overcome with respect to its governance. Recent policymaking has focused on establishing principles for the trustworthy use of AI. Adhering to these principles is especially important to protect vulnerable groups and ensure their confidence in the technology and its uses. Taking the perspective of Veterans, we focus on three principles of particular interest: (i) designing, developing, acquiring, and using AI where the benefits of use significantly outweigh the risks and the risks are assessed and managed, (ii) ensuring that the application of AI occurs in well-defined domains and are accurate, effective, and fit for intended purposes, and (iii) ensure the operations and outcomes of AI applications are sufficiently interpretable and understandable by all subject matter experts, users, and others. We also explain how these three principles build upon an existing ethics framework within the Department of Veterans Affairs and how the VA can continue to modernize its technology governance to simultaneously leverage the gains of AI and manage its risks.

Using Machine Learning to Understand Veterans' Receipt of Loans in the Paycheck Protection Program

The COVID-19 pandemic had a profound impact on small businesses. One of the policy responses was the passage of the Paycheck Protection Program (PPP), aimed at helping businesses stay afloat and maintaining their payroll. The NAII examined how the PPP affected Veteran entrepreneurs and found that Veteran entrepreneurs were more likely to receive loans and, conditional on receiving a loan, received larger loans. Machine learning tools were utilized to predict the receipt of PPP funds and found that local Veteran Affairs medical centers (VAMCs) played an important role. The results suggest that, in addition to treating patients, VAMCs help with the socialization of information.

Using Machine Learning to Understand Veterans' Receipt of Loans in the Paycheck Protection Program

This paper provides the first quantitative investigation of the receipt of funds from the Paycheck Protection Program (PPP) among Veterans between April and June. We find that Veterans received 3.5% more loans and 6.8% larger loans than their counterparts (p<0.01), controlling for a wide array of zip code characteristics and exploits within-zip code variation in further robustness. We subsequently use machine learning to predict PPP loan receipt among Veterans, finding that characteristics about quality of the local Department of Veterans Affairs medical centers are predictive. We develop models to predict the number of PPP loans awarded to Veteran-owned, finding that the inclusion of local VA medical center characteristics adds almost as much explanatory power as the industry and occupational composition in an area and even more than the education, race, and age distribution combined. Our results suggest that VA medical centers can play an essential role in helping Veterans thrive even beyond addressing their direct medical needs.



For collaboration opportunities, contact us at naii@va.gov.











Questions about the R&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.