相关文章推荐
文武双全的企鹅  ·  声音大师课| ...·  2 月前    · 
温柔的黄花菜  ·  Think-Cell - Student ...·  4 月前    · 
爱看书的卤蛋  ·  騰訊WeChat ...·  4 月前    · 
冷冷的红薯  ·  The Renaissance_百度百科·  1 年前    · 

set_index caused ValueError: Length mismatch #4255

@mrweijianli

Description

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04) : MacOS BigSur 11.2.3
  • Modin version ( modin.__version__ ): 0.13.2
  • Python version : Python 3.9.7
  • Code we can use to reproduce :
  • Describe the problem

    When I used set_index to set a MultiIndex from columns of the dataframe, the following ValueError happened:

    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    

    Source code / logs

    The dataset and the types are the same as that in another issue: #4254

    The two lines of code that caused this problem are:

    data1 = data1.reset_index()
    data1 = data1.set_index(['ric', 'source', 'date'])
    

    I did some experiments and made sure that the same operations worked in pandas.

    ---------------------------------------------------------------------------
    RayTaskError(ValueError)                  Traceback (most recent call last)
    Input In [8], in <module>
    ----> 1 dc.compare_diff_ratio_for_all_rics()
    Input In [6], in DataCompare.compare_diff_ratio_for_all_rics(self)
         31 data2['source'] = '2'
         32 data1 = data1.reset_index()
    ---> 34 data1 = data1.set_index(['ric', 'source', 'date'])
         35 data2 = data2.reset_index()
         36 data2 = data2.set_index(['ric', 'source', 'date'])
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/pandas/dataframe.py:1951, in DataFrame.set_index(self, keys, drop, append, inplace, verify_integrity)
       1949     else:
       1950         return
    -> 1951 new_query_compiler = self._query_compiler.set_index_from_columns(
       1952     keys, drop=drop, append=append
       1953 )
       1955 if verify_integrity and not new_query_compiler.index.is_unique:
       1956     duplicates = new_query_compiler.index[
       1957         new_query_compiler.index.duplicated()
       1958     ].unique()
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/storage_formats/pandas/query_compiler.py:613, in PandasQueryCompiler.set_index_from_columns(self, keys, drop, append)
        610 def set_index_from_columns(
        611     self, keys: List[Hashable], drop: bool = True, append: bool = False
        612 ):
    --> 613     new_modin_frame = self._modin_frame.to_labels(keys)
        614     if append:
        615         arrays = []
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py:840, in PandasDataframe.to_labels(self, column_list)
        826 def to_labels(self, column_list: List[Hashable]) -> "PandasDataframe":
        827     """
        828     Move one or more columns into the row labels. Previous labels are dropped.
       (...)
        838         A new PandasDataframe that has the updated labels.
        839     """
    --> 840     extracted_columns = self.mask(col_labels=column_list).to_pandas()
        841     if len(column_list) == 1:
        842         new_labels = pandas.Index(extracted_columns.squeeze(axis=1))
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py:110, in lazy_metadata_decorator.<locals>.decorator.<locals>.run_f_on_minimally_updated_metadata(self, *args, **kwargs)
        108     elif apply_axis == "rows":
        109         obj._propagate_index_objs(axis=0)
    --> 110 result = f(self, *args, **kwargs)
        111 if apply_axis is None and not transpose:
        112     result._deferred_index = self._deferred_index
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py:2699, in PandasDataframe.to_pandas(self)
       2690 @lazy_metadata_decorator(apply_axis="both")
       2691 def to_pandas(self):
       2692     """
       2693     Convert this Modin DataFrame to a pandas DataFrame.
       (...)
       2697     pandas.DataFrame
       2698     """
    -> 2699     df = self._partition_mgr_cls.to_pandas(self._partitions)
       2700     if df.empty:
       2701         df = pandas.DataFrame(columns=self.columns, index=self.index)
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/partitioning/partition_manager.py:632, in PandasDataframePartitionManager.to_pandas(cls, partitions)
        617 @classmethod
        618 def to_pandas(cls, partitions):
        619     """
        620     Convert NumPy array of PandasDataframePartition to pandas DataFrame.
       (...)
        630         A pandas DataFrame
        631     """
    --> 632     retrieved_objects = [[obj.to_pandas() for obj in part] for part in partitions]
        633     if all(
        634         isinstance(part, pandas.Series) for row in retrieved_objects for part in row
        635     ):
        636         axis = 0
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/partitioning/partition_manager.py:632, in <listcomp>(.0)
        617 @classmethod
        618 def to_pandas(cls, partitions):
        619     """
        620     Convert NumPy array of PandasDataframePartition to pandas DataFrame.
       (...)
        630         A pandas DataFrame
        631     """
    --> 632     retrieved_objects = [[obj.to_pandas() for obj in part] for part in partitions]
        633     if all(
        634         isinstance(part, pandas.Series) for row in retrieved_objects for part in row
        635     ):
        636         axis = 0
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/partitioning/partition_manager.py:632, in <listcomp>(.0)
        617 @classmethod
        618 def to_pandas(cls, partitions):
        619     """
        620     Convert NumPy array of PandasDataframePartition to pandas DataFrame.
       (...)
        630         A pandas DataFrame
        631     """
    --> 632     retrieved_objects = [[obj.to_pandas() for obj in part] for part in partitions]
        633     if all(
        634         isinstance(part, pandas.Series) for row in retrieved_objects for part in row
        635     ):
        636         axis = 0
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py:192, in PandasOnRayDataframePartition.to_pandas(self)
        184 def to_pandas(self):
        185     """
        186     Convert the object wrapped by this partition to a ``pandas.DataFrame``.
       (...)
        190     pandas DataFrame.
        191     """
    --> 192     dataframe = self.get()
        193     assert type(dataframe) is pandas.DataFrame or type(dataframe) is pandas.Series
        194     return dataframe
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py:74, in PandasOnRayDataframePartition.get(self)
         72 if len(self.call_queue):
         73     self.drain_call_queue()
    ---> 74 return ray.get(self.oid)
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/ray/_private/client_mode_hook.py:105, in client_mode_hook.<locals>.wrapper(*args, **kwargs)
        103     if func.__name__ != "init" or is_client_mode_enabled_by_default:
        104         return getattr(ray, func.__name__)(*args, **kwargs)
    --> 105 return func(*args, **kwargs)
    File ~/opt/anaconda3/envs/py39/lib/python3.9/site-packages/ray/worker.py:1733, in get(object_refs, timeout)
       1731     worker.core_worker.dump_object_store_memory_usage()
       1732 if isinstance(value, RayTaskError):
    -> 1733     raise value.as_instanceof_cause()
       1734 else:
       1735     raise value
    RayTaskError(ValueError): ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    During handling of the above exception, another exception occurred:
    ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py", line 426, in apply_func
        result = func(partition.copy(), *args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    2022-02-22 15:50:28,005	ERROR worker.py:85 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    During handling of the above exception, another exception occurred:
    ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py", line 426, in apply_func
        result = func(partition.copy(), *args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    2022-02-22 15:50:28,007	ERROR worker.py:85 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    During handling of the above exception, another exception occurred:
    ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py", line 426, in apply_func
        result = func(partition.copy(), *args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    2022-02-22 15:50:28,008	ERROR worker.py:85 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    During handling of the above exception, another exception occurred:
    ray::apply_func() (pid=12327, ip=127.0.0.1)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py", line 426, in apply_func
        result = func(partition.copy(), *args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/modin/core/dataframe/pandas/dataframe/dataframe.py", line 499, in apply_idx_objs
        return df.set_axis(idx, axis="index", inplace=False)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 764, in _set_axis_nocheck
        obj.set_axis(labels, axis=axis, inplace=True)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
        return func(*args, **kwargs)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/frame.py", line 4785, in set_axis
        return super().set_axis(labels, axis=axis, inplace=inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 755, in set_axis
        return self._set_axis_nocheck(labels, axis, inplace)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 761, in _set_axis_nocheck
        setattr(self, self._get_axis_name(axis), labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 5596, in __setattr__
        return object.__setattr__(self, name, value)
      File "pandas/_libs/properties.pyx", line 70, in pandas._libs.properties.AxisProperty.__set__
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/generic.py", line 769, in _set_axis
        self._mgr.set_axis(axis, labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 214, in set_axis
        self._validate_set_axis(axis, new_labels)
      File "/Users/weijianli/opt/anaconda3/envs/py39/lib/python3.9/site-packages/pandas/core/internals/base.py", line 64, in _validate_set_axis
        raise ValueError(
    ValueError: Length mismatch: Expected axis has 267314 elements, new values have 267309 elements
    

    The Length mismatch error above repeats indefinitely unless I restart the jupyter notebook kernel.