# SFTTrainer Loss function

**URL:** <https://discuss.huggingface.co/t/sfttrainer-loss-function/80549>\
**Category:** Beginners\
**Created:** [April 7, 2024, 11:51am UTC](https://discuss.huggingface.co/t/sfttrainer-loss-function/80549 "2024-04-07T11:51:10Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [July 8, 2024, 8:12am UTC](https://discuss.huggingface.co/t/sfttrainer-loss-function/80549/3 "2024-07-08T08:12:55Z")

</div>

> [@ZeyadMahmoud](#):
>
> - how to know the loss function used by default for SFTTrainer for a given model and how to alter it?

The loss function being used is the cross-entropy loss. It is defined within the model, e.g. [here](https://github.com/huggingface/transformers/blob/1082361a1978d30db5c3932d1ee08914d74d9697/src/transformers/models/llama/modeling_llama.py#L1202) for llama. In case you want to use your own custom loss function, you can overwrite the compute\_loss method of the Trainer as explained [here](https://discuss.huggingface.co/t/finetuning-bart-using-custom-loss/4060/2).

> 1. For training an LLM the loss function is computed on the whole concatenated prompts, how to alter this and make loss function only compute on the output prompts

Indeed as recommended above, the [DataCollatorForCompletionOnlyLM](https://huggingface.co/docs/trl/en/sft_trainer#train-on-completions-only) can be used for this purpose.

---

_[View the full topic](https://discuss.huggingface.co/t/sfttrainer-loss-function/80549)._