A deep learning pipeline for de novo molecular generation using LSTM and reinforcement learning (REINFORCE), with support for fragment-based control and automated molecule evaluation.

This project enables the generation of novel, valid, and drug-like molecules as SMILES strings, using a two-stage approach:

• Stage 1: Train an LSTM model on a large SMILES dataset for next-token prediction.
• Stage 2: Fine-tune the model with reinforcement learning (REINFORCE) using a custom reward function (validity, novelty, fragment presence, QED).

• Robust SMILES preprocessing and tokenization
• LSTM sequence modeling
• RL fine-tuning with chemical property rewards
• Top-p (nucleus) sampling for diverse molecule generation
• Automated evaluation: validity, novelty, diversity, QED, uniqueness
• Visualization of generated molecules

.
├── data/                           # Raw .smi files
│   ├── CCAB.smi
│   ├── CCAB.pkl
│   └── README.md
│
├── src/
│   ├── preprocess.py               # Data preprocessing script
│   ├── tokenize_smiles.py          # Tokenized SMILES and dictionaries
│   ├── split_data_tokens.py        # Data splits for training
│   ├── train_model.py              # LSTM model training
│   ├── RLfinetune.py               # RL fine-tuning script
│   ├── testRLLSTM.py               # Generation and evaluation
│   ├── test.py                     # Utility script for data testing
│   ├── comparison.py               # Compares baseline LSTM vs RL fine-tuned model results
│   ├── molwtdist.py                # Plots molecular weight distribution of generated molecules
│   ├── plot.py                     # Generates training and reward progression plots
│   └── RL+LSTM_model.ipynb         # Jupyter notebook (pipeline demo)
│
├── results/ 
│   ├── generated_smiles_CN.smi     # Generated SMILES with validity
│   ├── README.md
│   └── figures/
│       ├── training_loss_accuracy.png
│       ├── cumulative_rewards.png
|       ├── generated_smiles_CN.png  # Image grid of valid molecules  
│       └── (other generated plots)
│
├── Documentation Report.pdf
├── requirements.txt                # Python dependencies
└── README.md
          

pip install -r requirements.txt

• Python 3.10
• TensorFlow 2.19.0
• RDKit 2024.9.6
• NumPy
• Pandas
• Matplotlib

python preprocess.py

• Output: processed_data/preprocessing_step_1.smi

python tokenize_smiles.py

• Output: processed_data/tokenized_data.pkl

python split_data_tokens.py

• Output: 10 splits in processed_data/split_data/

python train_model.py

• Output: saved_model/Orig/lstm_generator.h5

python RLfinetune.py

• Output: saved_model/Orig/lstm_finetuned_rl.h5, reward logs, and plot

python testRLLSTM.py

• Outputs:
  • generated_smiles_CN.smi: Generated SMILES with validity flags
  • generated_molecules_CN.png: Grid image of valid molecules
  • Console: Evaluation metrics (validity, novelty, diversity, QED, uniqueness)

Summary: Out of 100 generated molecules, 97 were chemically valid, all were novel, and the set exhibited high diversity and uniqueness. The average QED (quantitative estimate of drug-likeness) was 0.62, indicating good drug-like properties.

we have file in codes folder please look over there

If you use this codebase or results, please cite this repository and the relevant RDKit and TensorFlow libraries.

For questions or contributions, please open an issue or contact the maintainer.

Note: This pipeline is modular and can be adapted for different chemical fragment targets or reward functions as needed.