hubert-base-ls960-speech-commands-h

This model is a fine-tuned version of facebook/hubert-base-ls960 on the speech_commands dataset. It achieves the following results on the evaluation set:

• Loss: 1.3148
• Accuracy: 0.7594

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 48
• eval_batch_size: 48
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.743	1.0	824	3.4107	0.1781
2.3383	2.0	1648	3.4632	0.1862
2.2702	3.0	2472	3.5701	0.0787
2.3059	4.0	3296	3.5742	0.0971
2.2574	5.0	4120	3.5457	0.1493
2.0617	6.0	4944	2.8490	0.3453
2.0289	7.0	5768	2.7607	0.3215
1.7807	8.0	6592	2.5721	0.4681
1.8188	9.0	7416	2.5625	0.5301
1.3812	10.0	8240	2.4258	0.6942
1.3136	11.0	9064	2.2087	0.6884
1.2867	12.0	9888	1.8347	0.7221
1.1036	13.0	10712	1.6731	0.7383
0.9534	14.0	11536	1.8732	0.7307
0.9289	15.0	12360	1.5742	0.7415
1.0973	16.0	13184	1.3693	0.7365
0.989	17.0	14008	1.2718	0.7455
0.8876	18.0	14832	1.3148	0.7594
0.814	19.0	15656	1.2231	0.7558
0.9899	20.0	16480	1.2349	0.7522

Framework versions

• Transformers 4.43.3
• Pytorch 2.2.2+cu121
• Datasets 2.18.0
• Tokenizers 0.19.1