最近对于本地私有化模型，我是通过seldon这一套进行封装，也就是最终是通过mlserver启动模型服务，然后外部调用mlserver的接口去做模型的推理。

其中调用mlserver的推理服务的时候，涉及到传输参数的序列化，序列化的大概样例如下：

tools: list = None
df=pd.Series(
            [
                tools
            ],
            index=[     
                "tools"
            ],
        )
PandasCodec.encode_request(df, use_bytes=False)

其中PandasCodec是mlserver提供的序列化类，此代码在mlserver 1.4.0以前的版本下都没有问题，在1.4.0开始，包括1.4.0会出现如下问题：

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

跟踪代码会发现：

https://github.com/SeldonIO/MLServer/blob/release/1.4.0/mlserver/codecs/numpy.py#L108

对于dataframe中的每列，会做是否非空的检查：

def _to_response_output(series: pd.Series, use_bytes: bool = True) -> ResponseOutput:
    datatype = to_datatype(series.dtype)
    data = series.tolist()

    # Replace NaN with null
    has_nan = series.isnull().any()
    if has_nan:
        data = list(map(convert_nan, data))

convert_nan的逻辑为：

def convert_nan(val):
    if np.isnan(val):
        return None

    return val

由于np的isnan是：

The `isnan` method doesn't work on Numpy arrays with non-numeric

这就意味着，对于前面的tools来说，是non-numeric类型，通过调试也可以看到，tools的类型为BYTES，这就导致np.isnan(val)直接抛出异常。

由于mlserver.1.4.0之前并无此校验，因此之前的代码不会有问题。

https://github.com/SeldonIO/MLServer/issues/1873 首先给他们挂了个issue描述了一下问题。

通过追踪mlserver的commit记录，可以看到：

https://github.com/SeldonIO/MLServer/commit/be8bab5938b478d68f3ac0da895a5e8af73c2586

在这个commit后，新增了这段逻辑，按照seldon自己的描述：

The NaN (Not a Number) value is used in Numpy and other scientific libraries to
describe an invalid or missing value (e.g. a division by zero).
In some scenarios, it may be desirable to let your models receive and / or
output NaN values (e.g. these can be useful sometimes with GBTs, like XGBoost
models).
This is why MLServer supports encoding NaN values on your request / response
payloads under some conditions.

In order to send / receive NaN values, you must ensure that:

- You are using the `REST` interface.
- The input / output entry containing NaN values uses either the `FP16`, `FP32`
  or `FP64` datatypes.
- You are either using the [Pandas codec](#pandas-dataframe) or the [Numpy
  codec](#numpy-array).

Assuming those conditions are satisfied, any `null` value within your tensor
payload will be converted to NaN.

For example, if you take the following Numpy array:

归纳下来就是因为如果对于非基本类型接收None，那么在某些模型下，比如XGBoost就会出现模型输出NAN，为了解决这个问题，就不再支持None。

搞清楚了原因，那么解决方案就明确了，按照官方的描述，既然他们是这样认定，基本上后续的版本中也不会做额外的处理。

于是我自己重写了一个codecs，重写了PandasCodec的方法：

from typing import List

import numpy as np
import pandas as pd
from mlserver.codecs import PandasCodec
from mlserver.codecs.numpy import to_datatype
from mlserver.codecs.pandas import _process_bytes
from mlserver.codecs.utils import inject_batch_dimension
from mlserver.types import InferenceRequest, Parameters, RequestInput, ResponseOutput, Datatype


def convert_nan(val):
    try:
        if np.isnan(val):
            return None
    except Exception:
        return val
    return val


def _to_response_output(series: pd.Series, use_bytes: bool = True) -> ResponseOutput:
    datatype = to_datatype(series.dtype)
    data = series.tolist()

    # Replace NaN with null
    has_nan = series.isnull().any()
    if has_nan:
        data = list(map(convert_nan, data))

    content_type = None
    if datatype == Datatype.BYTES:
        data, content_type = _process_bytes(data, use_bytes)

    shape = inject_batch_dimension(list(series.shape))
    parameters = None
    if content_type:
        parameters = Parameters(content_type=content_type)

    return ResponseOutput(
        name=series.name,
        shape=shape,
        data=data,
        datatype=datatype,
        parameters=parameters,
    )


class SeldonPandasCodec(PandasCodec):
    @classmethod
    def encode_outputs(
            cls, payload: pd.DataFrame, use_bytes: bool = True
    ) -> List[ResponseOutput]:
        return [
            _to_response_output(payload[col], use_bytes=use_bytes) for col in payload
        ]

    @classmethod
    def encode_request(
            cls, payload: pd.DataFrame, use_bytes: bool = True, **kwargs
    ) -> InferenceRequest:
        outputs = cls.encode_outputs(payload, use_bytes=use_bytes)

        return InferenceRequest(
            parameters=Parameters(content_type=cls.ContentType),
            inputs=[
                RequestInput(
                    name=output.name,
                    datatype=output.datatype,
                    shape=output.shape,
                    data=output.data,
                    parameters=output.parameters,
                )
                for output in outputs
            ],
        )

针对isnan的判断做了额外的处理，使用的时候由PandasCodec.encode_request换成SeldonPandasCodec.encode_request即可。

扫码手机观看或分享：