Spaces:

Hazem
/

Fac256xc

Runtime error

App Files Files Community

Fac256xc / tools /train /train.py

Commit History

fix(train): overwrite dropout only when specified

89bc9d4

boris commited on May 5, 2022

feat: allow eval only

65bb95f

boris commited on May 5, 2022

feat(train): allow editing dropout during training

80d791a

boris commited on May 3, 2022

feat: support LR offset (#174)

c3e93df
unverified

boris commited on Apr 29, 2022

feat(train): allow nesterov momentum

d08bf8d

boris commited on Apr 29, 2022

feat: allow weight decay

acc1a4a

boris commited on Apr 28, 2022

feat(train): use new HF _do_init api

6b84155

boris commited on Apr 19, 2022

fix: apply learning rate offset only when requested

c6263f3

boris commited on Apr 16, 2022

feat(train): arg to offset lr for resumed runs

89b4c45

boris commited on Apr 16, 2022

fix: allow non-scanned models (#168)

8ae9176
unverified

boris commited on Apr 15, 2022

feat: vmap optimizer (#166)

b993d27
unverified

boris commited on Apr 11, 2022

feat: scan layers + gradient checkpointing (#161)

07a6f9a
unverified

boris commited on Apr 9, 2022

feat: better multi-node support (#158)

728a3c3
unverified

boris commited on Apr 8, 2022

feat(model): allow bias (#152)

361a994
unverified

boris commited on Mar 30, 2022

feat(train): google-cloud-storage is optional

02b2308

boris commited on Mar 29, 2022

feat(train): rename logged config

955dc20

boris commited on Mar 29, 2022

feat: implement transformer variants (#144)

542378c
unverified

boris commited on Mar 24, 2022

feat(train): log norm and histograms (#143)

b7b619a
unverified

boris commited on Mar 19, 2022

feat(data): super conditioning (#141)

7939874
unverified

boris commited on Mar 17, 2022

feat: support pod (#139)

803ccbf
unverified

boris commited on Mar 14, 2022

fix: no gradient checkpointing for new model

2e02683

boris commited on Feb 23, 2022

feat: no gradient checkpointing for params init

b798ed3

boris commited on Feb 23, 2022

fix(train): consider schedule offset

bc4734f

boris commited on Feb 23, 2022

feat(train): local jax cache

9f5e879

boris commited on Feb 23, 2022

feat: add bucket reference to artifact

d368fb6

boris commited on Feb 23, 2022

style: lint

d5d442a

boris commited on Feb 22, 2022

feat: handle gradient checkpointing

5173ec7

boris commited on Feb 22, 2022

feat: load from bucket

1c4e839

boris commited on Feb 22, 2022

feat(train): save to bucket

50498e6

boris commited on Feb 22, 2022

feat: reduce artifact space + offset step

34cf91c

boris commited on Feb 22, 2022

feat: restore weights on CPU

5f954fc

boris commited on Feb 21, 2022

feat(train): simplify tokenizer loading

4cb21dd

boris commited on Feb 21, 2022

feat(train): use compilation cache

da9367c

boris commited on Feb 7, 2022

feat: log num_parameters early

7cfe576

boris commited on Feb 2, 2022

feat(train) - handle multiple nodes (#130)

0952927
unverified

boris commited on Feb 1, 2022

feat: handle model parallel

1bb3269

boris commited on Feb 1, 2022

feat(train): more custom x-axis

5f28cd2

boris commited on Jan 31, 2022

fix(train): opt_state_shape for distributed_shampoo

225b6ff

boris commited on Jan 29, 2022

feat(train): split artifact into model/state

fa5b058

boris commited on Jan 27, 2022

feat(train): another 25% faster

14abe8c

boris commited on Jan 26, 2022

feat(train): overhead from 70% to 1% 🥳

2b7f5f1

boris commited on Jan 26, 2022

feat(pjit): follow t5x style

7b5868f

boris commited on Jan 26, 2022

fix(train): grads spec

00710bc

boris commited on Jan 25, 2022

feat(train): improve pjit speed

f254058

boris commited on Jan 25, 2022

fix(train): consider correct batch size

b7c7458

boris commited on Jan 25, 2022

feat(train): custom start_preconditioning_step

8149924

boris commited on Jan 24, 2022

feat(train): handle distributed_shampoo in pjit

032f623

boris commited on Jan 24, 2022

feat(train): distributed_shampoo with pjit

cc34d07

boris commited on Jan 24, 2022

fix style

f044cb8

boris commited on Jan 23, 2022

feat(train): restore opt_state efficiently

1bfc1b5

boris commited on Jan 23, 2022