Policy Considerations for Artificial Intelligence in Public Health: Privacy, Equity, and Accountability

Artificial intelligence can help public health agencies respond faster and more accurately to emerging threats. But without strong policy, its use could become unsafe, inequitable, and untrustworthy.

Why policy matters for AI in public health

Public health agencies are facing a crisis. They are expected to detect, interpret, and respond to threats faster and with more precision than ever before, while operating with fewer resources, diminished authority, and increasing public scrutiny. Artificial intelligence (AI) has the potential to help with this crisis, by accelerating the collection and processing of data, improving the accuracy of analyses, and tailoring communications for diverse audiences.

AI is a general-purpose technology (like electricity or the internet) whose capabilities, risks, and social implications extend well beyond any single use case. Without clear policy, its integration into public health could be inconsistent, inequitable, and unsafe.

AI policy for public health must serve two simultaneous goals:

1. Enable innovation so agencies can harness AI’s speed, scale, and analytic power to improve population health.
2. Guard against the risks to privacy, fairness, security, and trust that can undermine the legitimacy of public health action.

Building on legal, ethical, and operational foundations for AI in public health

Legal mandates

Most state and local public health agencies operate under laws that require specific diseases, conditions, or events to be reported to the agency. These laws often empower the agency to collect identifiable health data without consent, but only for defined purposes related to disease prevention and control. Federal statutes such as HIPAA generally do not restrict these activities, but state laws and regulations impose strict controls on who can access the data, how it can be stored, and when it can be shared.

Ethical principles

Because people are often legally required to provide information to public health agencies, there is an ethical duty to protect it with the highest possible standards. This obligation extends beyond preventing data breaches. It includes ensuring that data are used only for legitimate purposes, that analyses are free from bias, and that the results are communicated in ways that do not cause unnecessary harm to individuals or communities.

Public health data is different than other data

In public health, every data point exists in a political, cultural, and economic context. Decisions about when to act, how to communicate, and what resources to deploy are filtered through constraints like budget limits, workforce shortages, public perception, and the priorities of elected officials. AI systems that take these realities into account will deliver more value.

Core policy domains for AI in public health

Data governance

AI systems depend on data, and, in public health, data only flows because of laws and trust.

• Define the categories of data that can be used for AI development and operations, including whether personally identifiable information may be included.
• Require data minimization, so AI systems use only the fields necessary for their function.
• Set rules for data linkage, such as when surveillance data may be combined with clinical, environmental, or social service datasets.
• Specify retention periods for training data, logs, and outputs, ensuring compliance with state archival rules and privacy standards.

Agencies should maintain data inventories and lineage tracking so that any AI model’s outputs can be traced back to their original data sources.

Procurement and vendor oversight

Public health agencies often lack the internal capacity to develop AI systems entirely in-house, making vendor partnerships inevitable. Without strong procurement policies, agencies risk locking themselves into proprietary systems that are expensive to maintain, opaque in operation, or incompatible with future needs.

Key policy provisions should include:

• Transparency requirements: Vendors must disclose the data sources, model architectures, and training procedures used.
• Security and residency guarantees: All data must be stored and processed in secure environments, preferably within U.S. jurisdiction, and never used for unrelated purposes without explicit approval.
• Exit strategies: Contracts should guarantee that the agency retains rights to its data and can transfer them to another system without disruption if the vendor relationship ends.
• Testing before deployment: AI tools should be piloted on historical or synthetic datasets before being used in live operations.

Access control and auditability

• Role-based access controls are enforced within AI interfaces, matching the permissions already in place for human users of source databases.
• Every interaction with the AI system is logged, including the user ID, date and time, query submitted, and data accessed.
• Audit logs are reviewed regularly to detect anomalies, such as repeated access to data outside a user’s normal scope.

Model validation and bias mitigation

• Pre-deployment testing against a gold-standard dataset to establish baseline performance.
• Periodic revalidation after updates or retraining.
• Bias audits to detect whether outputs differ systematically across demographic groups, geographic areas, or other population segments.
• Mechanisms for documenting and addressing identified biases.

Human oversight and accountability

• Who is responsible for validating AI outputs.
• How disagreements between AI recommendations and human judgment are resolved.
• What documentation is needed to justify final decisions.

Communication and transparency

• Public documentation of AI use cases, capabilities, and limitations.
• Plain-language summaries of how models work and what data they use.
• Clear disclaimers when outputs are provisional or based on incomplete data.

Technical safeguards for privacy, equity, and security with AI in public health

Secure enclaves and closed networks

Where possible, AI models handling sensitive health data should run within secure computing environments controlled by the public health agency. External connections should be minimized and monitored.

Differential privacy

This technique injects statistical noise into datasets or outputs to make it mathematically unlikely to re-identify individuals, while preserving overall patterns for analysis.

Federated learning

Instead of pooling all data in a central repository, federated learning allows AI models to be trained locally on separate datasets, with only the model parameters (not the underlying data) shared for aggregation.

Provenance tracking

Both data and models should be version-controlled, so the agency can reconstruct exactly which inputs and algorithms produced a given output.

Access anomaly detection

Automated monitoring can flag unusual access patterns, such as a user suddenly downloading large volumes of data or repeatedly querying sensitive subsets.

Navigating the politics of public health

• Building bipartisan support for core privacy and equity principles.
• Engaging community stakeholders early, so they understand and can vouch for the safeguards in place.
• Establishing independent review boards or ethics committees to provide credible oversight and reduce the perception that AI decisions are purely bureaucratic or politically driven.

Consequences of wrong policies in public health

If AI integration in public health proceeds without coherent policy, the consequences could be severe. A data breach involving an AI vendor could erode public trust for years. A biased model could misallocate resources away from communities that need them most. An opaque decision process could invite legal challenges that slow or block urgent interventions.

Public health agencies need to act now to implement AI policy

State and local public health agencies do not need to wait for a national AI policy framework to begin. Many of the elements (data governance, access control, procurement oversight, bias auditing, and transparent communication) can be adapted from existing rules for other sensitive information systems.

Artificial intelligence will not replace the fundamental work of public health. With thoughtful, enforceable policies, it can become a trusted partner that amplifies human capacity while honoring the legal and ethical commitments that are foundational to public health.