Motif-Technologies
/

activation

Model card Files Files and versions

activation / tests

35.5 kB

Ctrl+K

Ctrl+K

9 contributors

History: 15 commits

TaehyunKim

feat: add GroupedFusedMulPolyNorm Triton kernel for MoE models (#16)

e195bbb unverified 15 days ago

__init__.py

0 Bytes
feat(poly-norm): Add PolyNorm 10 months ago
allclose_default.py

512 Bytes
Add fusion (#3) 6 months ago
pytest.ini

45 Bytes
fix(poly-norm): calc param grad explicitly 10 months ago
test_fused_add_rms_norm.py

3.6 kB
refactor(activation): change fused_add_rms_norm and fused_add_rms_norm_backward to out-place operations 5 months ago
test_fused_add_rms_norm_sequence_parallel.py

5.85 kB
fix: fix fused add rms norm sharding strategy 4 months ago
test_fused_mul_poly_norm.py

4.13 kB
Add fusion (#3) 6 months ago
test_grouped_fused_mul_poly_norm.py

9.94 kB
feat: add GroupedFusedMulPolyNorm Triton kernel for MoE models (#16) 15 days ago
test_poly_norm.py

2.55 kB
Add fusion (#3) 6 months ago
test_rms_norm.py

1.94 kB
feat: make rms_norm as out-place 6 months ago
test_rms_norm_sequence_parallel.py

4.37 kB
fix: fix rms norm sharding strategy 4 months ago
utils.py

2.56 kB
Add fusion (#3) 6 months ago