Test failure with Python 3.13.0b1 #533

frenzymadness · 2024-05-13T11:40:27Z

This assertion fails:

Line 113 in f111f7a

assert list(clsdict.keys()) == ["C_CONSTANT", "__doc__", "method_c"]

with the following error:

=================================== FAILURES ===================================
___________________________ test_extract_class_dict ____________________________

    def test_extract_class_dict():
        class A(int):
            """A docstring"""
    
            def method(self):
                return "a"
    
        class B:
            """B docstring"""
    
            B_CONSTANT = 42
    
            def method(self):
                return "b"
    
        class C(A, B):
            C_CONSTANT = 43
    
            def method_c(self):
                return "c"
    
        clsdict = _extract_class_dict(C)
>       assert sorted(clsdict.keys()) == ["C_CONSTANT", "__doc__", "method_c"]
E       AssertionError: assert ['C_CONSTANT'...', 'method_c'] == ['C_CONSTANT'...', 'method_c']
E         At index 2 diff: '__firstlineno__' != 'method_c'
E         Left contains one more item: 'method_c'
E         Full diff:
E         - ['C_CONSTANT', '__doc__', 'method_c']
E         + ['C_CONSTANT', '__doc__', '__firstlineno__', 'method_c']
E         ?                          +++++++++++++++++++

tests/cloudpickle_test.py:111: AssertionError
=========================== short test summary info ============================
FAILED tests/cloudpickle_test.py::test_extract_class_dict - AssertionError: a...
============ 1 failed, 232 passed, 25 skipped, 2 warnings in 11.75s ============

This change in Python seems to be related: python/cpython#118475

The text was updated successfully, but these errors were encountered:

Resolves: cloudpipe#533

frenzymadness · 2024-05-15T08:56:34Z

Hmm, we have the test failing with Python 3.13 beta 1 and cloudpickle 3.0.0 and it's true that this problem is fixed by the proposed PR but when I run all the tests with cloudpickle from the main branch, I'm getting more errors:

========================================= FAILURES ==========================================
____ CloudPickleTest.test_deterministic_dynamic_class_attr_ordering_for_chained_pickling ____

self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_deterministic_dynamic_class_attr_ordering_for_chained_pickling>

    def test_deterministic_dynamic_class_attr_ordering_for_chained_pickling(self):
        # Check that the pickle produced by pickling a reconstructed class definition
        # in a remote process matches the pickle produced by pickling the original
        # class definition.
        # In particular, this test checks that the order of the class attributes is
        # deterministic.
    
        with subprocess_worker(protocol=self.protocol) as w:
    
            class A:
                """Simple class definition"""
    
                pass
    
            A_dump = w.run(cloudpickle.dumps, A)
            check_deterministic_pickle(A_dump, cloudpickle.dumps(A))
    
            # If the `__doc__` attribute is defined after some other class
            # attribute, this can cause class attribute ordering changes due to
            # the way we reconstruct the class definition in
            # `_make_class_skeleton`, which creates the class and thus its
            # `__doc__` attribute before populating the class attributes.
            class A:
                name = "A"
                __doc__ = "Updated class definition"
    
            A_dump = w.run(cloudpickle.dumps, A)
            check_deterministic_pickle(A_dump, cloudpickle.dumps(A))
    
            # If a `__doc__` is defined on the `__init__` method, this can
            # cause ordering changes due to the way we reconstruct the class
            # with `_make_class_skeleton`.
            class A:
                def __init__(self):
                    """Class definition with explicit __init__"""
                    pass
    
            A_dump = w.run(cloudpickle.dumps, A)
>           check_deterministic_pickle(A_dump, cloudpickle.dumps(A))

A          = <class 'tests.cloudpickle_test.CloudPickleTest.test_deterministic_dynamic_class_attr_ordering_for_chained_pickling.<locals>.A'>
A_dump     = b"\x80\x05\x95\x14\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
self       = <tests.cloudpickle_test.CloudPickleTest testMethod=test_deterministic_dynamic_class_attr_ordering_for_chained_pickling>
w          = <tests.testutils._Worker object at 0x7f46db632510>

tests/cloudpickle_test.py:2090: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = b"\x80\x05\x95\x14\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
b = b"\x80\x05\x95\xac\x03\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."

    def check_deterministic_pickle(a, b):
        """Check that two pickle output are bitwise equal.
    
        If it is not the case, print the diff between the disassembled pickle
        payloads.
    
        This helper is useful to investigate non-deterministic pickling.
        """
        if a != b:
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(a), out)
                a_out = out.getvalue()
                # Remove the 11 first characters of each line to remove the bytecode offset
                # of each object, which is different on each line for very small differences,
                # making the diff very hard to read.
                a_out = "\n".join(ll[11:] for ll in a_out.splitlines())
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(b), out)
                b_out = out.getvalue()
                b_out = "\n".join(ll[11:] for ll in b_out.splitlines())
>           assert a_out == b_out
E           AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
E             
E                PROTO      5
E             -  FRAME      879
E             ?              ^^
E             +  FRAME      981
E             ?             + ^
E                SHORT_BINUNICODE 'cloudpickle.cloudpickle'...
E             
E             ...Full output truncated (150 lines hidden), use '-vv' to show

a          = b"\x80\x05\x95\x14\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
a_out      = " PROTO      5\n FRAME      981\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '_...PTY_TUPLE\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
b          = b"\x80\x05\x95\xac\x03\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
b_out      = " PROTO      5\n FRAME      879\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '_...PTY_TUPLE\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
out        = <_io.StringIO object at 0x7f46db6ad6c0>

tests/testutils.py:239: AssertionError
____ CloudPickleTest.test_deterministic_str_interning_for_chained_dynamic_class_pickling ____

self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_deterministic_str_interning_for_chained_dynamic_class_pickling>

    def test_deterministic_str_interning_for_chained_dynamic_class_pickling(self):
        # Check that the pickle produced by the unpickled instance is the same.
        # This checks that there is no issue related to the string interning of
        # the names of attributes of class definitions and names of attributes
        # of the `__code__` objects of the methods.
    
        with subprocess_worker(protocol=self.protocol) as w:
            # Due to interning of class attributes, check that this does not
            # create issues with dynamic function definition.
            class A:
                """Class with potential string interning issues."""
    
                arg_1 = "class_value"
    
                def join(self):
                    pass
    
                def test_method(self, arg_1, join):
                    pass
    
            A_dump = w.run(cloudpickle.dumps, A)
>           check_deterministic_pickle(A_dump, cloudpickle.dumps(A))

A          = <class 'tests.cloudpickle_test.CloudPickleTest.test_deterministic_str_interning_for_chained_dynamic_class_pickling.<locals>.A'>
A_dump     = b'\x80\x05\x95\x94\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...ass_pickling.<locals>.A.test_method\x94h<}\x94h>Nh?Nh\x0bh\x0ch@NhANhB]\x94hD}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
self       = <tests.cloudpickle_test.CloudPickleTest testMethod=test_deterministic_str_interning_for_chained_dynamic_class_pickling>
w          = <tests.testutils._Worker object at 0x7f46db410690>

tests/cloudpickle_test.py:2113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = b'\x80\x05\x95\x94\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...ass_pickling.<locals>.A.test_method\x94h<}\x94h>Nh?Nh\x0bh\x0ch@NhANhB]\x94hD}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
b = b'\x80\x05\x95\xc5\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...\x94}\x94(h0\x8c\x0btest_method\x94h:hLh;}\x94h=Nh>Nh\x0bh\x0ch?Nh@NhA]\x94hC}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'

    def check_deterministic_pickle(a, b):
        """Check that two pickle output are bitwise equal.
    
        If it is not the case, print the diff between the disassembled pickle
        payloads.
    
        This helper is useful to investigate non-deterministic pickling.
        """
        if a != b:
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(a), out)
                a_out = out.getvalue()
                # Remove the 11 first characters of each line to remove the bytecode offset
                # of each object, which is different on each line for very small differences,
                # making the diff very hard to read.
                a_out = "\n".join(ll[11:] for ll in a_out.splitlines())
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(b), out)
                b_out = out.getvalue()
                b_out = "\n".join(ll[11:] for ll in b_out.splitlines())
>           assert a_out == b_out
E           AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
E             
E                PROTO      5
E             -  FRAME      1158
E             ?               --
E             +  FRAME      1361
E             ?              ++
E                SHORT_BINUNICODE 'cloudpickle.cloudpickle'...
E             
E             ...Full output truncated (288 lines hidden), use '-vv' to show

a          = b'\x80\x05\x95\x94\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...ass_pickling.<locals>.A.test_method\x94h<}\x94h>Nh?Nh\x0bh\x0ch@NhANhB]\x94hD}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
a_out      = " PROTO      5\n FRAME      1361\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '...n     POP\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
b          = b'\x80\x05\x95\xc5\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...\x94}\x94(h0\x8c\x0btest_method\x94h:hLh;}\x94h=Nh>Nh\x0bh\x0ch?Nh@NhA]\x94hC}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
b_out      = " PROTO      5\n FRAME      1158\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '...n     POP\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
out        = <_io.StringIO object at 0x7f46db6add80>

tests/testutils.py:239: AssertionError
________ CloudPickleTest.test_dynamic_class_determinist_subworker_tuple_memoization _________

self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_dynamic_class_determinist_subworker_tuple_memoization>

    def test_dynamic_class_determinist_subworker_tuple_memoization(self):
        # Check that the pickle produced by the unpickled instance is the same.
        # This highlights some issues with tuple memoization.
    
        with subprocess_worker(protocol=self.protocol) as w:
            # Arguments' tuple is memoized in the main process but not in the
            # subprocess as the tuples do not share the same id in the loaded
            # class.
    
            # XXX - this does not seem to work, and I am not sure there is an easy fix.
            class A:
                """Class with potential tuple memoization issues."""
    
                def func1(self):
                    pass
    
                def func2(self):
                    pass
    
            A_dump = w.run(cloudpickle.dumps, A)
>           check_deterministic_pickle(A_dump, cloudpickle.dumps(A))

A          = <class 'tests.cloudpickle_test.CloudPickleTest.test_dynamic_class_determinist_subworker_tuple_memoization.<locals>.A'>
A_dump     = b'\x80\x05\x953\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\x94..._tuple_memoization.<locals>.A.func2\x94h:}\x94h<Nh=Nh\x0bh\x0ch>Nh?Nh@]\x94hB}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
self       = <tests.cloudpickle_test.CloudPickleTest testMethod=test_dynamic_class_determinist_subworker_tuple_memoization>
w          = <tests.testutils._Worker object at 0x7f46db6f6c10>

tests/cloudpickle_test.py:2153: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = b'\x80\x05\x953\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\x94..._tuple_memoization.<locals>.A.func2\x94h:}\x94h<Nh=Nh\x0bh\x0ch>Nh?Nh@]\x94hB}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
b = b'\x80\x05\x95{\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\x94...4h4hL}\x94}\x94(h.\x8c\x05func2\x94h8hHh9}\x94h;Nh<Nh\x0bh\x0ch=Nh>Nh?]\x94hA}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'

    def check_deterministic_pickle(a, b):
        """Check that two pickle output are bitwise equal.
    
        If it is not the case, print the diff between the disassembled pickle
        payloads.
    
        This helper is useful to investigate non-deterministic pickling.
        """
        if a != b:
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(a), out)
                a_out = out.getvalue()
                # Remove the 11 first characters of each line to remove the bytecode offset
                # of each object, which is different on each line for very small differences,
                # making the diff very hard to read.
                a_out = "\n".join(ll[11:] for ll in a_out.splitlines())
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(b), out)
                b_out = out.getvalue()
                b_out = "\n".join(ll[11:] for ll in b_out.splitlines())
>           assert a_out == b_out
E           AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
E             
E                PROTO      5
E             -  FRAME      1088
E             ?              ^^
E             +  FRAME      1268
E             ?              ^^
E                SHORT_BINUNICODE 'cloudpickle.cloudpickle'...
E             
E             ...Full output truncated (284 lines hidden), use '-vv' to show

a          = b'\x80\x05\x953\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\x94..._tuple_memoization.<locals>.A.func2\x94h:}\x94h<Nh=Nh\x0bh\x0ch>Nh?Nh@]\x94hB}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
a_out      = " PROTO      5\n FRAME      1268\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '...n     POP\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
b          = b'\x80\x05\x95{\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\x94...4h4hL}\x94}\x94(h.\x8c\x05func2\x94h8hHh9}\x94h;Nh<Nh\x0bh\x0ch=Nh>Nh?]\x94hA}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
b_out      = " PROTO      5\n FRAME      1088\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '...n     POP\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
out        = <_io.StringIO object at 0x7f46db6ae080>

tests/testutils.py:239: AssertionError
_ Protocol2CloudPickleTest.test_deterministic_dynamic_class_attr_ordering_for_chained_pickling _

self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_deterministic_dynamic_class_attr_ordering_for_chained_pickling>

    def test_deterministic_dynamic_class_attr_ordering_for_chained_pickling(self):
        # Check that the pickle produced by pickling a reconstructed class definition
        # in a remote process matches the pickle produced by pickling the original
        # class definition.
        # In particular, this test checks that the order of the class attributes is
        # deterministic.
    
        with subprocess_worker(protocol=self.protocol) as w:
    
            class A:
                """Simple class definition"""
    
                pass
    
            A_dump = w.run(cloudpickle.dumps, A)
            check_deterministic_pickle(A_dump, cloudpickle.dumps(A))
    
            # If the `__doc__` attribute is defined after some other class
            # attribute, this can cause class attribute ordering changes due to
            # the way we reconstruct the class definition in
            # `_make_class_skeleton`, which creates the class and thus its
            # `__doc__` attribute before populating the class attributes.
            class A:
                name = "A"
                __doc__ = "Updated class definition"
    
            A_dump = w.run(cloudpickle.dumps, A)
            check_deterministic_pickle(A_dump, cloudpickle.dumps(A))
    
            # If a `__doc__` is defined on the `__init__` method, this can
            # cause ordering changes due to the way we reconstruct the class
            # with `_make_class_skeleton`.
            class A:
                def __init__(self):
                    """Class definition with explicit __init__"""
                    pass
    
            A_dump = w.run(cloudpickle.dumps, A)
>           check_deterministic_pickle(A_dump, cloudpickle.dumps(A))

A          = <class 'tests.cloudpickle_test.CloudPickleTest.test_deterministic_dynamic_class_attr_ordering_for_chained_pickling.<locals>.A'>
A_dump     = b"\x80\x05\x95\x14\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
self       = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_deterministic_dynamic_class_attr_ordering_for_chained_pickling>
w          = <tests.testutils._Worker object at 0x7f46d9f6ff20>

tests/cloudpickle_test.py:2090: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = b"\x80\x05\x95\x14\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
b = b"\x80\x05\x95\xac\x03\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."

    def check_deterministic_pickle(a, b):
        """Check that two pickle output are bitwise equal.
    
        If it is not the case, print the diff between the disassembled pickle
        payloads.
    
        This helper is useful to investigate non-deterministic pickling.
        """
        if a != b:
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(a), out)
                a_out = out.getvalue()
                # Remove the 11 first characters of each line to remove the bytecode offset
                # of each object, which is different on each line for very small differences,
                # making the diff very hard to read.
                a_out = "\n".join(ll[11:] for ll in a_out.splitlines())
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(b), out)
                b_out = out.getvalue()
                b_out = "\n".join(ll[11:] for ll in b_out.splitlines())
>           assert a_out == b_out
E           AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
E             
E                PROTO      5
E             -  FRAME      879
E             ?              ^^
E             +  FRAME      981
E             ?             + ^
E                SHORT_BINUNICODE 'cloudpickle.cloudpickle'...
E             
E             ...Full output truncated (150 lines hidden), use '-vv' to show

a          = b"\x80\x05\x95\x14\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
a_out      = " PROTO      5\n FRAME      981\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '_...PTY_TUPLE\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
b          = b"\x80\x05\x95\xac\x03\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...x0b__globals__\x94}\x94u\x86\x94\x86R0\x8c\n__module__\x94h\x0c\x8c\x15__static_attributes__\x94)u}\x94\x86\x94\x86R0."
b_out      = " PROTO      5\n FRAME      879\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '_...PTY_TUPLE\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
out        = <_io.StringIO object at 0x7f46d88cee00>

tests/testutils.py:239: AssertionError
_ Protocol2CloudPickleTest.test_deterministic_str_interning_for_chained_dynamic_class_pickling _

self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_deterministic_str_interning_for_chained_dynamic_class_pickling>

    def test_deterministic_str_interning_for_chained_dynamic_class_pickling(self):
        # Check that the pickle produced by the unpickled instance is the same.
        # This checks that there is no issue related to the string interning of
        # the names of attributes of class definitions and names of attributes
        # of the `__code__` objects of the methods.
    
        with subprocess_worker(protocol=self.protocol) as w:
            # Due to interning of class attributes, check that this does not
            # create issues with dynamic function definition.
            class A:
                """Class with potential string interning issues."""
    
                arg_1 = "class_value"
    
                def join(self):
                    pass
    
                def test_method(self, arg_1, join):
                    pass
    
            A_dump = w.run(cloudpickle.dumps, A)
>           check_deterministic_pickle(A_dump, cloudpickle.dumps(A))

A          = <class 'tests.cloudpickle_test.CloudPickleTest.test_deterministic_str_interning_for_chained_dynamic_class_pickling.<locals>.A'>
A_dump     = b'\x80\x05\x95\x94\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...ass_pickling.<locals>.A.test_method\x94h<}\x94h>Nh?Nh\x0bh\x0ch@NhANhB]\x94hD}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
self       = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_deterministic_str_interning_for_chained_dynamic_class_pickling>
w          = <tests.testutils._Worker object at 0x7f46d845faf0>

tests/cloudpickle_test.py:2113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = b'\x80\x05\x95\x94\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...ass_pickling.<locals>.A.test_method\x94h<}\x94h>Nh?Nh\x0bh\x0ch@NhANhB]\x94hD}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
b = b'\x80\x05\x95\xc5\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...\x94}\x94(h0\x8c\x0btest_method\x94h:hLh;}\x94h=Nh>Nh\x0bh\x0ch?Nh@NhA]\x94hC}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'

    def check_deterministic_pickle(a, b):
        """Check that two pickle output are bitwise equal.
    
        If it is not the case, print the diff between the disassembled pickle
        payloads.
    
        This helper is useful to investigate non-deterministic pickling.
        """
        if a != b:
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(a), out)
                a_out = out.getvalue()
                # Remove the 11 first characters of each line to remove the bytecode offset
                # of each object, which is different on each line for very small differences,
                # making the diff very hard to read.
                a_out = "\n".join(ll[11:] for ll in a_out.splitlines())
            with io.StringIO() as out:
                pickletools.dis(pickletools.optimize(b), out)
                b_out = out.getvalue()
                b_out = "\n".join(ll[11:] for ll in b_out.splitlines())
>           assert a_out == b_out
E           AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
E             
E                PROTO      5
E             -  FRAME      1158
E             ?               --
E             +  FRAME      1361
E             ?              ++
E                SHORT_BINUNICODE 'cloudpickle.cloudpickle'...
E             
E             ...Full output truncated (288 lines hidden), use '-vv' to show

a          = b'\x80\x05\x95\x94\x05\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...ass_pickling.<locals>.A.test_method\x94h<}\x94h>Nh?Nh\x0bh\x0ch@NhANhB]\x94hD}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
a_out      = " PROTO      5\n FRAME      1361\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '...n     POP\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
b          = b'\x80\x05\x95\xc5\x04\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x14_make_skeleton_class\x94\x93\...\x94}\x94(h0\x8c\x0btest_method\x94h:hLh;}\x94h=Nh>Nh\x0bh\x0ch?Nh@NhA]\x94hC}\x94u\x86\x94\x86R0u}\x94\x86\x94\x86R0.'
b_out      = " PROTO      5\n FRAME      1158\n SHORT_BINUNICODE 'cloudpickle.cloudpickle'\n MEMOIZE    (as 0)\n SHORT_BINUNICODE '...n     POP\n     SETITEMS   (MARK at 195)\n EMPTY_DICT\n TUPLE2\n TUPLE2\n REDUCE\n POP\n STOP\ntocol among opcodes = 4"
out        = <_io.StringIO object at 0x7f46d88cf580>

tests/testutils.py:239: AssertionError
===================================== warnings summary ======================================
tests/cloudpickle_test.py: 12 warnings
  /usr/lib64/python3.13/multiprocessing/popen_fork.py:67: DeprecationWarning: This process (pid=62090) is multi-threaded, use of fork() may lead to deadlocks in the child.
    self.pid = os.fork()

tests/cloudpickle_test.py::CloudPickleTest::test_itertools_count
  /home/lbalhar/Software/cloudpickle/cloudpickle/cloudpickle.py:1284: DeprecationWarning: Pickle, copy, and deepcopy support will be removed from itertools in Python 3.14.
    return super().dump(obj)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================== short test summary info ==================================
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_deterministic_dynamic_class_attr_ordering_for_chained_pickling - AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_deterministic_str_interning_for_chained_dynamic_class_pickling - AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_dynamic_class_determinist_subworker_tuple_memoization - AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_deterministic_dynamic_class_attr_ordering_for_chained_pickling - AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_deterministic_str_interning_for_chained_dynamic_class_pickling - AssertionError: assert ' PROTO      ...g opcodes = 4' == ' PROTO      ...g opcodes = 4'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 5 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============= 5 failed, 156 passed, 4 skipped, 1 xfailed, 13 warnings in 4.43s ==============
py313: exit 1 (4.62 seconds) /home/lbalhar/Software/cloudpickle> pytest -lv --maxfail=5 pid=62090

frenzymadness added a commit to frenzymadness/cloudpickle that referenced this issue May 15, 2024

Fix test_extract_class_dict for Python 3.13 beta 1

3c300a0

Resolves: cloudpipe#533

frenzymadness linked a pull request May 15, 2024 that will close this issue

Fix test_extract_class_dict for Python 3.13 beta 1 #534

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failure with Python 3.13.0b1 #533

Test failure with Python 3.13.0b1 #533

frenzymadness commented May 13, 2024

frenzymadness commented May 15, 2024

Test failure with Python 3.13.0b1 #533

Test failure with Python 3.13.0b1 #533

Comments

frenzymadness commented May 13, 2024

frenzymadness commented May 15, 2024