t5-small-finetuned-tf-xsum
This model was trained from scratch on xsum dataset.
It achieves the following results on the evaluation set:
- Train Loss: 2.3494
- Validation Loss: 2.1933
- Train Rouge1: 32.0241
- Train Rouge2: 10.1025
- Train Rougel: 25.8834
- Train Rougelsum: 25.9662
- Train Gen Len: 18.69
- Epoch: 8
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: float32
Training results
| Train Loss |
Validation Loss |
Train Rouge1 |
Train Rouge2 |
Train Rougel |
Train Rougelsum |
Train Gen Len |
Epoch |
| 2.7197 |
2.4028 |
29.6376 |
8.8596 |
22.8598 |
22.8401 |
18.82 |
1 |
| 2.5822 |
2.3407 |
30.6849 |
9.3100 |
23.8971 |
23.9096 |
18.745 |
2 |
| 2.5174 |
2.2979 |
32.3706 |
11.5463 |
26.4253 |
26.3525 |
18.75 |
3 |
| 2.4711 |
2.2703 |
32.2768 |
11.0460 |
26.2472 |
26.1540 |
18.825 |
4 |
| 2.4305 |
2.2432 |
29.3935 |
8.3337 |
22.2859 |
22.3557 |
18.65 |
5 |
| 2.3994 |
2.2237 |
31.0993 |
8.7932 |
23.6971 |
23.7702 |
18.815 |
6 |
| 2.3732 |
2.2071 |
31.4819 |
10.0677 |
25.1846 |
25.2829 |
18.675 |
7 |
| 2.3494 |
2.1933 |
32.0241 |
10.1025 |
25.8834 |
25.9662 |
18.69 |
8 |
Framework versions
- Transformers 4.21.1
- TensorFlow 2.8.2
- Datasets 2.4.0
- Tokenizers 0.12.1
