|
|
--- |
|
|
license: mit |
|
|
--- |
|
|
[](https://discord.gg/2JhHVh7CGu) |
|
|
|
|
|
A semi custom network based on the follow paper [Simpler Diffusion (SiD2)](https://arxiv.org/abs/2410.19324v1) |
|
|
|
|
|
This network uses the optimal transport flow matching objective outlined [Flow Matching for Generative Modeling](https://arxiv.org/abs/2210.02747) |
|
|
|
|
|
xATGLU Layers are used instead of linears for entry into the transformer MLP layer [Expanded Gating Ranges |
|
|
Improve Activation Functions](https://arxiv.org/pdf/2405.20768) |
|
|
|
|
|
 |
|
|
 |
|
|
 |
|
|
 |
