Highly accurate protein structure prediction with AlphaFold

Think of a protein sequence as a long sentence where each amino-acid “word” has a job. When we build a multiple-sequence alignment (MSA), we try to line sentences up so that the words that play the same part of the sentence—subject, verb, object—appear in vertical columns. In the protein world, that means:

When the alignment is correct, every row in a given column traces back to the same position in the common ancestor of all those proteins, so the residues in that column:

• Sit at the same 3-D spot in the folded structure, and
• Usually have the same job (catalysis, binding, packing, flexibility, etc.).

That’s why you often see:

• Conserved columns: the exact amino acid hardly changes because the role is intolerant to mutation (e.g., catalytic Ser).
• Variable but chemically similar columns: the individual letters differ, but they share properties—say, all hydrophobic—because any residue that behaves the same is good enough for that structural slot.

So in the sentence you quoted, “amino acids performing the same functional/structural role” means residues that occupy equivalent positions in homologous proteins and therefore contribute in equivalent ways to what the protein does or how it folds. Lining them up in the same column lets AlphaFold (or any analysis) read the evolutionary record of which jobs are critical and which can flex.

Example

TODO

Each cell gets an embedding vector; AlphaFold treats the first row (your query) as a special “single representation” and keeps the rest as contextual rows.

This tensor is the main input to the Evoformer block.

This geometric information is what eventually feeds into the pair representation and gives the network concrete spatial hints, especially in regions where the multiple‑sequence alignment (MSA) is thin.

E-value (expect value)

It measures a chance of meeting the sequence with aligment score bigger than given one in the whole database.
Formal definition

$E = Kmne^{-\lambda S}$ (the Karlin-Altschul equation), where S is the raw alignment score, m and n are the effective lengths of the query and database, and K, λ are statistical constants for the scoring matrix.

Lower is better. E = 1 × 10⁻³ means you’d expect one false-positive hit in a thousand equally good database searches.

Coverage threshold

$ \text{Coverage} = \frac{\text{aligned residues}}{\text{query length}}$. A spectacular E-value is useless if it aligns only a tiny fragment—that fragment may not be relevant to the overall fold you want to model.

Why both thresholds are used together

1. E-value guards against false positives (bad statistics).
2. Coverage guards against trivial positives (tiny true alignments that are uninformative for structure).

Only hits that pass both filters are promoted to “template” status; their atomic coordinates are then converted to the distance-and-orientation bins that seed the pair representation.

• What it does. For every single sequence (row) in the MSA, apply self‑attention along the residue axis.
• Why “pair bias”. The attention logits get an additive term derived from the current pair embedding for (i, j), so each sequence row is “aware” of the model’s latest guess about how residues i and j interact.
• Why “gated”. A learned sigmoid gate scales the output, letting the network suppress noisy signals when the alignment at that row is poor.
• Motivation. Propagate long‑range context within each homolog while conditioning on emerging structural hints.

• What it does. Now fix a residue position r and attend down the column across the s sequences.
• Motivation. Let each residue in the target sequence look at how that same position varies (or co‑varies) across evolution, capturing conservation and compensatory mutations.
• Complexity trick. Row‑ and column‑wise attention factorise a huge 2‑D attention ( s r × s r ) into two linear passes, cutting time and memory by ~O(s r)

• A two‑layer feed‑forward network (ReLU / GLU) applied to every (s, r) token independently — the same role a “Transformer FFN” plays after its attention layer.
• Purpose: mix features non‑linearly and give the model extra capacity. 

• How. For each residue pair $(i, j)$, take the embedding vectors $v_i, v_j$ from every sequence, form the outer product $v_i \bigotimes v_j$ (a matrix), then average over the $s$ sequences.
• What it produces. A rich co‑evolution feature saying “when residue $i$ mutates like this, residue $j$ mutates like that.”
• Where it goes. Added into the pair tensor, giving it evolutionary coupling signal.

• View the pair tensor as a fully‑connected directed graph of residues; each edge $(i, j)$ stores an embedding $z_{i, j}$.
• Update edge $(i, j)$ by multiplying / gating information that flows along the two edges $(i, k)$ and $(k, j)$ for every third residue $k$.
• Motivation: Impose triangle‑inequality‑like constraints and capture three‑body interactions essential for 3‑D geometry.

Algorithm

Let $\mathbf{Z} \in \mathbb{R}^{L \times L \times c}$ be the current pair tensor and let $i, j, k \in {1, \dots, L}$ index residues.

1. Project the two “arms” of each triangle

• $W^{(a)}, W^{(b)} \in \mathbb{R}^{c_m \times c}$ are learned linear maps.
• The projection width $c_m$ is usually $c/2$ so the outer product that follows keeps the channel count $\approx c$.

2. Form the triangle message for edge $(i, j)$:

• Element-wise product $\odot$ mixes the two arms.
• Division by $\sqrt{L}$ is a variance-stabilising scale (analogous to the $1/\sqrt{d}$ in self-attention).

3. Transform, gate and add as a residual

• $W^{(o)} \in \mathbb{R}^{c \times c_m}$ projects back to the original channel size.
• $g_{ij}$ is a sigmoid gate computed from the pre-update edge; it lets the network down-weight noisy triangles.
• The final line is a residual connection identical in spirit to the one in a Transformer block.

4. Vectorised Form

If we reshape $\mathbf{Z}$ to a 3-D tensor and use matrix multiplication, the whole operation is:

with $\mathbf{A} = W^{(a)} \mathbf{Z}, \quad \mathbf{B} = W^{(b)} \mathbf{Z}$ and the sum over $k$ is implemented by a batched tensor contraction.

Intuition

• The update $\textbf{looks along paths}$ $i \rightarrow k \rightarrow j$ (“out-going” edges from $i$).
• It injects $\textit{multiplicative}$ three-body information, helping the network encode triangle inequalities and side-chain packing constraints—something plain pairwise couplings cannot capture.

Same operation but centred on the opposite vertex, so the model sees both orientations of every residue triplet.

• Treat all edges that emanate from residue i as the “tokens” of an attention layer; use edge $(i, j)$ as the query and the set { $(i, k)$ } as keys/values.
• Learns patterns like “edges from this residue fan out in a β‑sheet vs. coil”. 

• Same again but for edges that end at residue j. Together, the two triangle‑attention layers let the model reason about local geometry from both directions. 

• A second feed‑forward network, now on every $(i, j)$ edge, to consolidate the information distilled by the triangle operations before looping to the next Evoformer block. 

IPA is the SE(3)‑equivariant replacement for the dot‑product attention you know from Transformers. The scalars (queries Q, keys K, values V) work almost exactly as usual, but every token (here: every residue i) is also equipped with a handful of small 3‑D point clouds that move rigidly with the residue’s current backbone frame.

A small MLP reads the new $S_i$ and predicts a delta rotation $\Delta R_i \in SO(3)$ (parameterised as a quaternion or axis–angle) and a delta translation $\Delta \mathbf{t}_i \in \mathbb{R}^3$.

Both deltas are passed through tanh-gated scales so that early iterations make only gentle moves; later passes can refine with larger steps.

After the 8-th shared block, a final head predicts \textbf{torsion angles} $\chi_1, \chi_2, \ldots$ per residue.

The angles are applied to idealised residue templates stored inside the network to produce \textbf{all heavy-atom coordinates} (including side chains and hydrogens).