ML engineering project
RNA 3D Folding Pipeline
RibonanzaNet is a deep learning model designed to predict the 3D structure of RNA molecules from their sequence alone. Understanding RNA structure is crucial for drug discovery and understanding biological processes. This project implements a dual-tower Transformer architecture that processes both biological (sequence/MSA) and geometric information. **Preprocessing & Features:** • **One-Hot Encoding**: The RNA sequence is converted into a numerical format using one-hot encoding, where each nucleotide (A, C, G, U) is represented as a 4-dimensional vector. $$ E = \{e_1, ..., e_L\} \quad \text{where} \quad e_i \in \{0,1\}^4 $$ • **Secondary Structure Prediction**: We compute a heuristic pairing score based on Watson-Crick (A-U, G-C) and Wobble (G-U) pairs, weighted by distance. This generates a base-pairing probability matrix used as an input feature. $$ S_{ij} = \text{score}(n_i, n_j) \times \max(0.1, 1.0 - \frac{j-i}{L}) $$ • **MSA Features**: Multiple Sequence Alignments (MSA) are processed to extract evolutionary information, including conservation scores (Shannon entropy) and covariance (mutual information) between residues. $$ H_i = -\sum_{b \in \{A,C,G,U\}} P_i(b) \log_2 P_i(b) $$ • **Geometric Augmentation**: To make the model robust to rotation, we apply random 3D rotations to the target coordinates during training. The rotation matrix $R$ is composed of rotations around the x, y, and z axes. $$ R = R_z(\theta_z) R_y(\theta_y) R_x(\theta_x) $$ **Model Architecture:** The model features a **Dual-Tower Architecture**: • **Biology Tower**: Processes sequence and MSA features using triangular attention to capture pairwise interactions. • **Geometry Tower**: Processes structural features and coordinates. • **Cross-Attention**: Fuses information between the two towers to refine the structural prediction. **Tech Stack:** Built with **Python**, **TensorFlow/Keras**, and **NumPy**. The pipeline handles FASTA parsing, MSA generation, and complex geometric transformations efficiently.
Problem
Predict RNA three-dimensional coordinates from sequence and evolutionary context without direct structural measurement.
My role
Built the preprocessing and modeling pipeline, including FASTA/MSA handling, geometric augmentation, dual encoders, and coordinate prediction.
Approach
Combine sequence, pairing, MSA, and geometry features in a dual-tower Transformer with cross-attention and rotation augmentation.
Outcome
Documented an end-to-end experimental RNA structure pipeline; no benchmark or deployment metric is claimed.
Implementation
- Encoded sequence, secondary-structure heuristics, conservation, and covariance features.
- Applied random 3D rotations to structural targets during training.
- Fused biology and geometry towers through cross-attention.
Evaluation
- Validated the data and model path as an experimental Kaggle notebook.
- No headline competition metric is present in the project record.
Results
- Produced sequence-to-coordinate predictions through a reproducible experimental pipeline.
- The result is presented as a modeling study rather than a validated scientific tool.
Constraints
- RNA sequences vary in length and have complex long-range relationships.
- Predictions must respect three-dimensional geometry and rotation invariance.
- Evolutionary MSA information may be sparse or expensive to prepare.
Tradeoffs
- Dual towers preserve modality structure but require more memory and coordination than one encoder.
- Heuristic pairing features are practical but do not replace experimentally derived structure.
Next improvements
- Add established RNA structure benchmarks and uncertainty reporting.
- Evaluate equivariant architectures and stronger geometric losses.