Commits

Commits on Aug 14, 2025

Fix a bug when Deepspeed-FPDT is disabled but with original Ulysses enabled (#479)

authored
aab2f31
add instruction document for huggingface UCP (#477)
Schwidola0607
authored
1d71682
fix the error issue for q/k/v stride is not match (#469)
yisustc
authored
8860868

Commits on Jul 16, 2025

add sequence_parallel in layernorm init to enable 3D parallelism can run successfully with DeepSpeed (#468)
yisustc
authored
4efb479

Commits on Feb 7, 2025

Update references to new GitHub org (deepspeedai) (#462)
loadams
authored
3e1da1f

Commits on Dec 24, 2024

Fix import error in `deepspeed_to_megatron.py` (#455)
hotsuyuki
authored
f4157be

Commits on Dec 18, 2024

[tool]GQA convert support (#454)
inkcherry
authored
c3df187

Commits on Dec 5, 2024

Adding the new feature of FPDT (#441)

authored
676a482

Commits on Oct 18, 2024

fix moe tflops (#445)
ranzhejiang
authored
6acc370

Commits on Oct 17, 2024

fix init issue for silently ignoring the deepspeed config (#452)
xylian86
authored
deb95cd

Commits on Oct 8, 2024

enable profiler for specific ranks (#451)
ranzhejiang
authored
4448492

Commits on Oct 4, 2024

fix init issue for rms_norm in squence_parallel (#448)
yisustc
authored
8be7f48

Commits on Sep 20, 2024

grad_wei can't be NoneType when running with DeepSpeed, for zero3 will divided the gradient (#428)
yisustc
authored
598c092

Commits on Sep 4, 2024

Enable Sequence Parallelism (#429)
Polisetty V R K Jyothendra Varma
authored
0d6e379

Commits on Aug 27, 2024

[Bug] Fix crash when logging optimizer state to tb (#417)
billishyahao
authored
1280f59
[wandb] disable wandb more gracefully (#422)

billishyahao
and
loadams
authored
543543a

Commits on Aug 26, 2024

enable empty cache on XPU device (#438)
yisustc
authored
8e9d973

Commits on Aug 24, 2024

fix nan issue when running megatron-deepspeed (#434)
yisustc
authored
4f9f1f6

Commits on Aug 19, 2024

fix TFLOPs calculation (#371)
Polisetty V R K Jyothendra Varma
authored
b7b2d5e

Commits on Aug 14, 2024

add device check when import ipex (#436)
yisustc
authored
cdf5194

Commits on Aug 10, 2024

[LLaMa] Adding support converting checkpoint from mds to hf (#432)
billishyahao
authored
f132876

Commits on Aug 7, 2024

pass batch_dim_idx to deepspeed sequence parallel distributed attention for supporting batch size larger than 1 (#433)

YJHMITWEB
and
Jinghan Yao
authored
61350c5

Commits on Aug 4, 2024

add fused_rms_norm support on XPU device (#431)
yisustc
authored
53b241f

Commits on Aug 1, 2024

fixing the bug of flash_attn import and the wrong gather index when using flash_attn_cuda in sequence parallel (#406)

YJHMITWEB
and
Jinghan Yao
authored
1bfc35c

Commits on Jul 29, 2024

Update/add GPT/Llama universal checkpointing scripts (#391)
lekurile
authored
8822a5c

Commits on Jul 19, 2024

Update yml to be valid (#427)
loadams
authored
3afd267
Add basic compilation test (#426)
loadams
authored
08b9376
acquire device when required (#420)
Polisetty V R K Jyothendra Varma
authored
3af2e25
improve repeat_kv GQA perf (#419)
Polisetty V R K Jyothendra Varma
authored
13f2673

Commits on Jul 18, 2024

add support to run custom Hf tokenizer for training and dataset pre-processing (#421)
Polisetty V R K Jyothendra Varma
authored
7d23e33

Commits on Jul 17, 2024

add kill switch file support to gracefully exit training at runtime (#412)
Polisetty V R K Jyothendra Varma
authored
fc989b8

Commits on Jul 16, 2024

[Wandb] Refine wandb logging function (#416)
billishyahao
authored
73029ed

Commits on Jul 15, 2024

add PyTorch profiler support (#414)
Polisetty V R K Jyothendra Varma
authored
0971e68

Commits on Jul 12, 2024

clear document (#395)
inkcherry
authored
73252c0
Extend test utilities to support more accelerators (#418)
xinyu-intel
authored
354e420