# Image Processor Image processor负责为视觉模型准备输入特征并后期处理处理它们的输出。这包括诸如调整大小、归一化和转换为PyTorch和NumPy张量等转换。它还可能包括特定于模型的后期处理，例如将logits转换为分割掩码。 ## ImageProcessingMixin[[transformers.ImageProcessingMixin]] #### transformers.ImageProcessingMixin[[transformers.ImageProcessingMixin]] [Source](https://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/image_processing_base.py#L61) This is an image processor mixin used to provide saving/loading functionality for sequential and image feature extractors. from_pretrainedtransformers.ImageProcessingMixin.from_pretrainedhttps://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/image_processing_base.py#L84[{"name": "pretrained_model_name_or_path", "val": ": typing.Union[str, os.PathLike]"}, {"name": "cache_dir", "val": ": typing.Union[str, os.PathLike, NoneType] = None"}, {"name": "force_download", "val": ": bool = False"}, {"name": "local_files_only", "val": ": bool = False"}, {"name": "token", "val": ": typing.Union[str, bool, NoneType] = None"}, {"name": "revision", "val": ": str = 'main'"}, {"name": "**kwargs", "val": ""}]- **pretrained_model_name_or_path** (`str` or `os.PathLike`) -- This can be either: - a string, the *model id* of a pretrained image_processor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a image processor file saved using the [save_pretrained()](/docs/transformers/v5.0.0rc2/zh/internal/image_processing_utils#transformers.ImageProcessingMixin.save_pretrained) method, e.g., `./my_model_directory/`. - a path or url to a saved image processor JSON *file*, e.g., `./my_model_directory/preprocessor_config.json`. - **cache_dir** (`str` or `os.PathLike`, *optional*) -- Path to a directory in which a downloaded pretrained model image processor should be cached if the standard cache should not be used. - **force_download** (`bool`, *optional*, defaults to `False`) -- Whether or not to force to (re-)download the image processor files and override the cached versions if they exist. - **proxies** (`dict[str, str]`, *optional*) -- A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. - **token** (`str` or `bool`, *optional*) -- The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). - **revision** (`str`, *optional*, defaults to `"main"`) -- The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. - **return_unused_kwargs** (`bool`, *optional*, defaults to `False`) -- If `False`, then this function returns just the final image processor object. If `True`, then this functions returns a `Tuple(image_processor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not image processor attributes: i.e., the part of `kwargs` which has not been used to update `image_processor` and is otherwise ignored. - **subfolder** (`str`, *optional*, defaults to `""`) -- In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. - **kwargs** (`dict[str, Any]`, *optional*) -- The values in kwargs of any keys which are image processor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* image processor attributes is controlled by the `return_unused_kwargs` keyword parameter.0A image processor of type [ImageProcessingMixin](/docs/transformers/v5.0.0rc2/zh/internal/image_processing_utils#transformers.ImageProcessingMixin). Instantiate a type of [ImageProcessingMixin](/docs/transformers/v5.0.0rc2/zh/internal/image_processing_utils#transformers.ImageProcessingMixin) from an image processor. Examples: ```python # We can't instantiate directly the base class *ImageProcessingMixin* so let's show the examples on a # derived class: *CLIPImageProcessor* image_processor = CLIPImageProcessor.from_pretrained( "openai/clip-vit-base-patch32" ) # Download image_processing_config from huggingface.co and cache. image_processor = CLIPImageProcessor.from_pretrained( "./test/saved_model/" ) # E.g. image processor (or model) was saved using *save_pretrained('./test/saved_model/')* image_processor = CLIPImageProcessor.from_pretrained("./test/saved_model/preprocessor_config.json") image_processor = CLIPImageProcessor.from_pretrained( "openai/clip-vit-base-patch32", do_normalize=False, foo=False ) assert image_processor.do_normalize is False image_processor, unused_kwargs = CLIPImageProcessor.from_pretrained( "openai/clip-vit-base-patch32", do_normalize=False, foo=False, return_unused_kwargs=True ) assert image_processor.do_normalize is False assert unused_kwargs == {"foo": False} ``` **Parameters:** pretrained_model_name_or_path (`str` or `os.PathLike`) : This can be either: - a string, the *model id* of a pretrained image_processor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a image processor file saved using the [save_pretrained()](/docs/transformers/v5.0.0rc2/zh/internal/image_processing_utils#transformers.ImageProcessingMixin.save_pretrained) method, e.g., `./my_model_directory/`. - a path or url to a saved image processor JSON *file*, e.g., `./my_model_directory/preprocessor_config.json`. cache_dir (`str` or `os.PathLike`, *optional*) : Path to a directory in which a downloaded pretrained model image processor should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`) : Whether or not to force to (re-)download the image processor files and override the cached versions if they exist. proxies (`dict[str, str]`, *optional*) : A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or `bool`, *optional*) : The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`) : The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. return_unused_kwargs (`bool`, *optional*, defaults to `False`) : If `False`, then this function returns just the final image processor object. If `True`, then this functions returns a `Tuple(image_processor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not image processor attributes: i.e., the part of `kwargs` which has not been used to update `image_processor` and is otherwise ignored. subfolder (`str`, *optional*, defaults to `""`) : In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. kwargs (`dict[str, Any]`, *optional*) : The values in kwargs of any keys which are image processor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* image processor attributes is controlled by the `return_unused_kwargs` keyword parameter. **Returns:** A image processor of type [ImageProcessingMixin](/docs/transformers/v5.0.0rc2/zh/internal/image_processing_utils#transformers.ImageProcessingMixin). #### save_pretrained[[transformers.ImageProcessingMixin.save_pretrained]] [Source](https://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/image_processing_base.py#L183) Save an image processor object to the directory `save_directory`, so that it can be re-loaded using the [from_pretrained()](/docs/transformers/v5.0.0rc2/zh/internal/image_processing_utils#transformers.ImageProcessingMixin.from_pretrained) class method. **Parameters:** save_directory (`str` or `os.PathLike`) : Directory where the image processor JSON file will be saved (will be created if it does not exist). push_to_hub (`bool`, *optional*, defaults to `False`) : Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`dict[str, Any]`, *optional*) : Additional key word arguments passed along to the [push_to_hub()](/docs/transformers/v5.0.0rc2/zh/main_classes/model#transformers.utils.PushToHubMixin.push_to_hub) method. ## BatchFeature[[transformers.BatchFeature]] #### transformers.BatchFeature[[transformers.BatchFeature]] [Source](https://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/feature_extraction_utils.py#L58) Holds the output of the [pad()](/docs/transformers/v5.0.0rc2/zh/main_classes/feature_extractor#transformers.SequenceFeatureExtractor.pad) and feature extractor specific `__call__` methods. This class is derived from a python dictionary and can be used as a dictionary. convert_to_tensorstransformers.BatchFeature.convert_to_tensorshttps://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/feature_extraction_utils.py#L157[{"name": "tensor_type", "val": ": typing.Union[str, transformers.utils.generic.TensorType, NoneType] = None"}, {"name": "skip_tensor_conversion", "val": ": typing.Union[list[str], set[str], NoneType] = None"}]- **tensor_type** (`str` or [TensorType](/docs/transformers/v5.0.0rc2/zh/internal/file_utils#transformers.TensorType), *optional*) -- The type of tensors to use. If `str`, should be one of the values of the enum [TensorType](/docs/transformers/v5.0.0rc2/zh/internal/file_utils#transformers.TensorType). If `None`, no modification is done. - **skip_tensor_conversion** (`list[str]` or `set[str]`, *optional*) -- List or set of keys that should NOT be converted to tensors, even when `tensor_type` is specified.0 Convert the inner content to tensors. Note: Values that don't have an array-like structure (e.g., strings, dicts, lists of strings) are automatically skipped and won't be converted to tensors. Ragged arrays (lists of arrays with different lengths) are still attempted, though they may raise errors during conversion. **Parameters:** data (`dict`, *optional*) : Dictionary of lists/arrays/tensors returned by the __call__/pad methods ('input_values', 'attention_mask', etc.). tensor_type (`Union[None, str, TensorType]`, *optional*) : You can give a tensor_type here to convert the lists of integers in PyTorch/Numpy Tensors at initialization. skip_tensor_conversion (`list[str]` or `set[str]`, *optional*) : List or set of keys that should NOT be converted to tensors, even when `tensor_type` is specified. #### to[[transformers.BatchFeature.to]] [Source](https://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/feature_extraction_utils.py#L211) Send all values to device by calling `v.to(*args, **kwargs)` (PyTorch only). This should support casting in different `dtypes` and sending the `BatchFeature` to a different `device`. **Parameters:** args (`Tuple`) : Will be passed to the `to(...)` function of the tensors. kwargs (`Dict`, *optional*) : Will be passed to the `to(...)` function of the tensors. To enable asynchronous data transfer, set the `non_blocking` flag in `kwargs` (defaults to `False`). **Returns:** `[BatchFeature](/docs/transformers/v5.0.0rc2/zh/main_classes/image_processor#transformers.BatchFeature)` The same instance after modification. ## BaseImageProcessor[[transformers.BaseImageProcessor]] #### transformers.BaseImageProcessor[[transformers.BaseImageProcessor]] [Source](https://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/image_processing_utils.py#L39) center_croptransformers.BaseImageProcessor.center_crophttps://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/image_processing_utils.py#L128[{"name": "image", "val": ": ndarray"}, {"name": "size", "val": ": dict"}, {"name": "data_format", "val": ": typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None"}, {"name": "input_data_format", "val": ": typing.Union[transformers.image_utils.ChannelDimension, str, NoneType] = None"}, {"name": "**kwargs", "val": ""}]- **image** (`np.ndarray`) -- Image to center crop. - **size** (`dict[str, int]`) -- Size of the output image. - **data_format** (`str` or `ChannelDimension`, *optional*) -- The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - **input_data_format** (`ChannelDimension` or `str`, *optional*) -- The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.0 Center crop an image to `(size["height"], size["width"])`. If the input size is smaller than `crop_size` along any edge, the image is padded with 0's and then center cropped. **Parameters:** image (`np.ndarray`) : Image to center crop. size (`dict[str, int]`) : Size of the output image. data_format (`str` or `ChannelDimension`, *optional*) : The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*) : The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. #### normalize[[transformers.BaseImageProcessor.normalize]] [Source](https://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/image_processing_utils.py#L91) Normalize an image. image = (image - image_mean) / image_std. **Parameters:** image (`np.ndarray`) : Image to normalize. mean (`float` or `Iterable[float]`) : Image mean to use for normalization. std (`float` or `Iterable[float]`) : Image standard deviation to use for normalization. data_format (`str` or `ChannelDimension`, *optional*) : The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*) : The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. **Returns:** ``np.ndarray`` The normalized image. #### rescale[[transformers.BaseImageProcessor.rescale]] [Source](https://github.com/huggingface/transformers/blob/v5.0.0rc2/src/transformers/image_processing_utils.py#L59) Rescale an image by a scale factor. image = image * scale. **Parameters:** image (`np.ndarray`) : Image to rescale. scale (`float`) : The scaling factor to rescale pixel values by. data_format (`str` or `ChannelDimension`, *optional*) : The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*) : The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. **Returns:** ``np.ndarray`` The rescaled image.