Configuring SFTP for HIPAA-Compliant EDI Transfers

The task is narrow and unforgiving: stand up an OpenSSH SFTP endpoint that accepts X12 837 claim batches and returns 835 remittances from named payers, while satisfying every HIPAA §164.312 technical safeguard for transmission integrity, access control, and audit — and a default sshd_config fails that bar on day one. This page is the concrete recipe: cipher policy, key-only authentication, per-payer isolation, PHI-safe async ingestion, and post-transfer integrity verification. It is the transport boundary that everything in Secure File Transfer Protocols for EDI sits on, and the first stage of the wider EDI Ingestion & Parsing Workflows pipeline.

Prerequisites

• OpenSSH 8.0+ on the server (for curve25519-sha256, ed25519 host keys, and -etm MACs)
• Python 3.10+ for the ingestion client (uses dataclass slots, AsyncIterator, and pathlib.Path typing)
• asyncssh for non-blocking SFTP (pip install asyncssh) — a third-party library, not part of the standard library
• aiofiles for non-blocking local writes (pip install aiofiles)
• A dedicated sftpusers system group and an unprivileged service account per trading partner — never share one account across payers
• Known X12 implementation-guide version per feed: 005010X222A2 for 837P inbound, 005010X221A1 for 835 return; the transport layer does not parse these but the directory layout should segregate them
• A pinned known_hosts entry for every payer you push to; inbound-only endpoints still need host keys distributed to senders out of band

Files that clear this transport boundary are handed straight to the parser — the same buffer discipline described in X12 Parser Performance Optimization takes over once staging completes.

Spec Reference: sshd_config Elements This Task Sets

The transport is only as strong as the weakest negotiated primitive. These directives pin the SSH transport to audited, modern suites and remove every legacy fallback a scanner (or an auditor) will flag.

The single most important row is ChrootDirectory: OpenSSH refuses to chroot into any path that is writable by a non-root user, so the chroot root itself must be owned by root:root at mode 0755 with the writable inbound/ subdirectory nested inside it.

Step-by-Step Implementation

Step 1 — Pin the SSH transport to audited primitives

Edit /etc/ssh/sshd_config (or a drop-in under /etc/ssh/sshd_config.d/) so negotiation can only land on suites aligned with the HIPAA Security Rule technical safeguards:

# /etc/ssh/sshd_config
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group16-sha512
Ciphers aes256-gcm@openssh.com,chacha20-poly1305@openssh.com
MACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com
HostKeyAlgorithms ssh-ed25519,rsa-sha2-512
AuthenticationMethods publickey
PasswordAuthentication no
PermitRootLogin no

Validate before reloading — a typo here can lock out every payer at once: sshd -t && systemctl reload ssh.

Step 2 — Isolate each payer with a root-owned chroot

Give every trading partner its own jail so a compromised or curious account cannot traverse into another payer’s 837 batches.

# One chroot per payer; root owns the jail, the account owns only inbound/
install -d -o root -g root -m 0755 /sftp/aetna
install -d -o svc_aetna -g sftpusers -m 0750 /sftp/aetna/inbound
install -d -o svc_aetna -g sftpusers -m 0750 /sftp/aetna/outbound

Bind the match block to the group so new payers inherit the policy:

Match Group sftpusers
    ChrootDirectory /sftp/%u
    ForceCommand internal-sftp -l INFO -f AUTH
    AllowTcpForwarding no
    X11Forwarding no

Step 3 — Stream the batch with bounded memory and PHI-safe logging

High-volume claim batches routinely exceed 2 GB, so a synchronous download plus in-memory read is an OOM waiting to happen. asyncssh with aiofiles reads the remote file in fixed chunks and never holds more than one chunk in memory. Every log line passes through a PHI mask first, satisfying the §164.312(b) requirement to keep patient identifiers and X12 control numbers out of the system-activity trail.

import asyncio
import asyncssh
import aiofiles
import logging
import re
from pathlib import Path
from typing import AsyncIterator
from dataclasses import dataclass

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")
logger = logging.getLogger("edi.sftp")

CHUNK_SIZE = 8 * 1024 * 1024  # 8 MB chunks bound peak memory on large 837 batches
MAX_RETRIES = 3
RETRY_DELAY = 2.0

_PHI_PATTERNS = [
    re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),        # SSN
    re.compile(r"\b\d{10,15}\b"),                 # MRN / account numbers
    re.compile(r"(ISA\d{2}|GS\d{2})\*[^*]*"),    # X12 control-number values (ISA13/GS06)
]

def mask_phi(message: str) -> str:
    """Redact PHI and X12 control numbers before anything reaches a log sink."""
    for pattern in _PHI_PATTERNS:
        message = pattern.sub("[REDACTED_PHI]", message)
    return message

@dataclass(slots=True)
class TransferConfig:
    host: str
    username: str
    key_path: Path
    known_hosts: Path                       # pin the payer host key; never None in production
    remote_path: str = "/inbound/payer_837_batch.dat"
    local_path: Path = Path("./staging/837_batch.dat")
    port: int = 22

async def stream_sftp_file(config: TransferConfig) -> AsyncIterator[bytes]:
    """Memory-bounded async SFTP reader with retry and error categorization."""
    attempt = 0
    while attempt < MAX_RETRIES:
        try:
            async with asyncssh.connect(
                config.host,
                port=config.port,
                username=config.username,
                client_keys=[str(config.key_path)],
                known_hosts=str(config.known_hosts),   # host-key verification is mandatory
                encryption_algs=["aes256-gcm@openssh.com", "chacha20-poly1305@openssh.com"],
            ) as conn:
                async with conn.start_sftp_client() as sftp:
                    async with await sftp.open(config.remote_path, "rb") as remote:
                        while chunk := await remote.read(CHUNK_SIZE):
                            yield chunk
            return
        except asyncssh.PermissionDenied as exc:
            logger.error(mask_phi(f"Auth failure: {exc}"))
            raise RuntimeError(
                "SFTP authentication failed. Verify key permissions and the payer allowlist."
            ) from exc
        except (asyncssh.ConnectionLost, ConnectionResetError) as exc:
            attempt += 1
            logger.warning(mask_phi(f"Connection interrupted (attempt {attempt}/{MAX_RETRIES}): {exc}"))
            if attempt >= MAX_RETRIES:
                raise
            await asyncio.sleep(RETRY_DELAY * attempt)
        except Exception as exc:  # noqa: BLE001 — categorize, then re-raise
            logger.error(mask_phi(f"Uncategorized SFTP error: {exc}"))
            raise

Step 4 — Stage the file and hand it to the pipeline

Decouple transport from parsing so a slow disk write never stalls the SSH connection, and delete partial files so the parser never sees a truncated interchange.

async def ingest_claim_batch(config: TransferConfig) -> Path:
    """Stream to local staging; on any failure, remove the partial file and re-raise."""
    config.local_path.parent.mkdir(parents=True, exist_ok=True)
    try:
        async with aiofiles.open(config.local_path, "wb") as local_file:
            async for chunk in stream_sftp_file(config):
                await local_file.write(chunk)
                await asyncio.sleep(0)  # yield so co-running EDI parsers keep progressing
        logger.info(mask_phi(f"Batch staged: {config.local_path.name}"))
        return config.local_path
    except Exception:
        if config.local_path.exists():
            config.local_path.unlink()  # never leave a truncated 837 for the parser
        raise

Step 5 — Verify integrity before the parser touches it

HIPAA §164.312©(1) requires a mechanism to confirm the data was not altered in transit. Compare a SHA-256 digest against the manifest the sender publishes alongside the batch.

import hashlib

def verify_digest(path: Path, expected_sha256: str) -> None:
    """Fail closed if the staged file does not match the sender's manifest."""
    digest = hashlib.sha256(path.read_bytes()).hexdigest()
    if digest != expected_sha256.lower():
        raise ValueError("SHA-256 mismatch — quarantine the batch; do not parse it.")
    logger.info(mask_phi(f"Integrity verified for {path.name}"))

Only after this check passes does the file cross into validation — routed through the Pydantic models for EDI schema validation that enforce the ISA → GS → ST envelope hierarchy, with transient and structural faults classified by the taxonomy in Error Categorization & Retry Logic Design.

Verification

Confirm the endpoint is both hardened and functional before pointing a payer at it:

• Cipher negotiation: ssh -vvv sftpuser@host 2>&1 | grep 'kex:' must show curve25519-sha256 and an AES-256-GCM cipher — never diffie-hellman-group1 or a CBC mode.
• Password auth is dead: ssh -o PreferredAuthentications=password sftpuser@host must return Permission denied (publickey) with no password prompt.
• Chroot holds: after login, pwd returns / and cd .. cannot escape /sftp/<payer>; a get from another payer’s path fails with No such file.
• Audit trail is clean: the SFTP subsystem log records open/close/write operations, and grep -E 'ISA13|[0-9]{3}-[0-9]{2}-[0-9]{4}' /var/log/auth.log returns nothing — the mask is working.
• Round trip: ingest_claim_batch() writes the staged file at flat memory, and verify_digest() logs Integrity verified against the sender’s manifest.

Common Gotchas

• ChrootDirectory permission denied. OpenSSH rejects any chroot path a non-root user can write to. Fix with chown root:root /sftp/<payer> && chmod 0755 /sftp/<payer>, and keep the writable inbound/ directory nested inside — this is the top support ticket for new payer onboarding.
• known_hosts=None in production. Disabling host-key verification for “brevity” removes the only defense against a man-in-the-middle on the SSH transport; pin the payer’s host key and treat any mismatch as a hard failure, exactly as secure file transfer protocols for EDI requires.
• PHI leaking into logs. Subsystem sftp -l INFO can echo filenames and, if you log raw exceptions, X12 control numbers (ISA13/GS06) or member IDs. Route every sink through mask_phi() and hash filenames that embed a claim identifier.
• Duplicate 837 submissions on retry. A reconnect can re-download and re-stage a batch already ingested; deduplicate downstream on a key of ISA13*GS06*ST02 before submission, mirroring the idempotency discipline in Error Categorization & Retry Logic Design.

Troubleshooting Reference

Compliance checklist (HIPAA §164.312):

• SSH host keys rotated annually; revoked keys removed from every authorized_keys.
• ChrootDirectory owned by root:root at 0755 — OpenSSH requirement and §164.312(a)(1) isolation.
• SFTP audit logs hashed (SHA-256) and written to WORM storage retained 6+ years.
• PHI masking applied to all application, system, and network log sinks (§164.312(b)).
• SHA-256 integrity check run post-transfer and pre-parse (§164.312©(1)).
• known_hosts pinning enabled; known_hosts=None permitted only in sandboxed tests.

Related

• Secure File Transfer Protocols for EDI — the transport architecture (SFTP, AS2, HTTPS/MFT) this configuration implements.
• Pydantic Models for EDI Schema Validation — the validation layer a staged, integrity-checked file enters next.
• Error Categorization & Retry Logic Design — the retry and idempotency taxonomy behind Steps 3–4.

Up next: return to Secure File Transfer Protocols for EDI for the surrounding transport model. For authoritative directive semantics see the OpenSSH sshd_config manual, and for the safeguards cited throughout, the HIPAA Security Rule.