Skip to content

[BUG] Multiple test failures after 69b400dc6ce7c709eb6ae7cf2963f50b5a73c656 #7908

Closed

Description

@antonio-rojas

Describe the bug

Commit 69b400d causes hundreds of test failures here. Posting just the first few ones:

Errors from shard 11:
======================================================================
FAIL: test_int_int (pycmp_typed)
Doctest: pycmp_typed.test_int_int
----------------------------------------------------------------------
AssertionError: Failed doctest test for pycmp_typed.test_int_int
  File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18736, in test_int_int

----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18740, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = 1; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, True, True, False, False)
Got:
    ERROR: test_int_int(0, 1) ==
    (False, False, True, True, True, True) !=
    (False, True, True, True, False, False)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18742, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = 2; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, True, True, False, False)
Got:
    ERROR: test_int_int(0, 2) ==
    (False, False, True, True, True, True) !=
    (False, True, True, True, False, False)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18744, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = -1; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, False, False, True, True)
Got:
    ERROR: test_int_int(0, -1) ==
    (False, False, False, False, False, False) !=
    (False, True, False, False, True, True)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18746, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = -2; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, False, False, True, True)
Got:
    ERROR: test_int_int(0, -2) ==
    (False, False, False, False, False, False) !=
    (False, True, False, False, True, True)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18748, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = 32768; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, True, True, False, False)
Got:
    ERROR: test_int_int(0, 32768) ==
    (False, False, True, True, True, True) !=
    (False, True, True, True, False, False)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18750, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = 32769; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, True, True, False, False)
Got:
    ERROR: test_int_int(0, 32769) ==
    (False, False, True, True, True, True) !=
    (False, True, True, True, False, False)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18752, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = -32768; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, False, False, True, True)
Got:
    ERROR: test_int_int(0, -32768) ==
    (False, False, False, False, False, False) !=
    (False, True, False, False, True, True)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18754, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = -32769; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, False, False, True, True)
Got:
    ERROR: test_int_int(0, -32769) ==
    (False, False, False, False, False, False) !=
    (False, True, False, False, True, True)
----------------------------------------------------------------------
File "/build/cython/src/cython/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18756, in pycmp_typed.test_int_int
Failed example:
    a = 0; b = 1073741824; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
Expected:
    (False, True, True, True, False, False)
Got:
    ERROR: test_int_int(0, 1073741824) ==
    (False, False, True, True, True, True) !=
    (False, True, True, True, False, False)

Code to reproduce the behaviour:

Compile 3.3.0, run python runtests.py -vv -j 64

Expected behaviour

Tests pass

OS

Arch Linux x86_64

Python version

3.14.7

Cython version

3.3.0

Additional context

No response

Activity

  1. da-woods commented on Aug 22, 2026

    @da-woods
    Collaborator

    I'm a bit puzzled by this because we run all these tests ourselves on every commit and the CI is green for us. So clearly something is different in your setup (although nothing is unusual based on your description)

  2. scoder commented on Aug 22, 2026

    @scoder
    Contributor

    What happens if you only run the single failing test module, i.e. python3 runtests.py pycmp_typed ?
    Could you also provide the output of the build_env test? It's meant to list relevant information about the test environment for debugging purposes.

  3. antonio-rojas commented on Aug 22, 2026

    @antonio-rojas
    Author

    What happens if you only run the single failing test module, i.e. python3 runtests.py pycmp_typed ?

    GNU gdb (GDB) 17.2
    Copyright (C) 2025 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Python 3.14.7 (main, Aug 10 2026, 07:46:56) [GCC 16.1.1 20260728]
    
    Running tests against Cython 3.3.0
    Using Cython language level 2.
    Backends: c,cpp
    
    runTest (__main__.CythonRunTestCase.runTest)
    [-1] compiling (c/cy2/pyx) and running pycmp_typed ... test_bytearray_bytearray (pycmp_typed)
    Doctest: pycmp_typed.test_bytearray_bytearray ... ok
    test_bytearray_bytes (pycmp_typed)
    Doctest: pycmp_typed.test_bytearray_bytes ... ok
    test_bytearray_bytesobject (pycmp_typed)
    Doctest: pycmp_typed.test_bytearray_bytesobject ... ok
    test_bytes_bytearray (pycmp_typed)
    Doctest: pycmp_typed.test_bytes_bytearray ... ok
    test_bytes_bytes (pycmp_typed)
    Doctest: pycmp_typed.test_bytes_bytes ... ok
    test_bytes_bytesobject (pycmp_typed)
    Doctest: pycmp_typed.test_bytes_bytesobject ... ok
    test_bytesobject_bytearray (pycmp_typed)
    Doctest: pycmp_typed.test_bytesobject_bytearray ... ok
    test_bytesobject_bytes (pycmp_typed)
    Doctest: pycmp_typed.test_bytesobject_bytes ... ok
    test_bytesobject_bytesobject (pycmp_typed)
    Doctest: pycmp_typed.test_bytesobject_bytesobject ... ok
    test_float_float (pycmp_typed)
    Doctest: pycmp_typed.test_float_float ... ok
    test_float_int (pycmp_typed)
    Doctest: pycmp_typed.test_float_int ... ok
    test_float_numobject (pycmp_typed)
    Doctest: pycmp_typed.test_float_numobject ... ok
    test_int_float (pycmp_typed)
    Doctest: pycmp_typed.test_int_float ... ok
    test_int_int (pycmp_typed)
    Doctest: pycmp_typed.test_int_int ... FAIL
    test_int_numobject (pycmp_typed)
    Doctest: pycmp_typed.test_int_numobject ... FAIL
    test_numobject_float (pycmp_typed)
    Doctest: pycmp_typed.test_numobject_float ... ok
    test_numobject_int (pycmp_typed)
    Doctest: pycmp_typed.test_numobject_int ... FAIL
    test_numobject_numobject (pycmp_typed)
    Doctest: pycmp_typed.test_numobject_numobject ... FAIL
    test_str_str (pycmp_typed)
    Doctest: pycmp_typed.test_str_str ... ok
    test_str_strobject (pycmp_typed)
    Doctest: pycmp_typed.test_str_strobject ... ok
    test_strobject_str (pycmp_typed)
    Doctest: pycmp_typed.test_strobject_str ... ok
    test_strobject_strobject (pycmp_typed)
    Doctest: pycmp_typed.test_strobject_strobject ... ok
    Test dependency found: 'numpy' version 2.5.2
    Test dependency not found: 'pythran'
    Test dependency not found: 'setuptools.sandbox'
    Test dependency found: 'asyncio' version 3.14.7
    Test dependency found: 'pstats' version 3.14.7
    Test dependency found: 'posix' version 3.14.7
    Test dependency found: 'array' version 3.14.7
    Test dependency not found: 'Cython.Coverage'
    Test dependency not found: 'Cython.Coverage'
    Test dependency not found: 'IPython.testing.globalipapp'
    Test dependency not found: 'jedi_BROKEN_AND_DISABLED'
    Test dependency found: 'test.support' version 3.14.7
    runTest (__main__.CythonRunTestCase.runTest)
    [-1] compiling (cpp/cy2/pyx) and running pycmp_typed ... 
    #### 2026-08-22 13:37:22.907863
    test_bytearray_bytearray (pycmp_typed)
    Doctest: pycmp_typed.test_bytearray_bytearray ... ok
    test_bytearray_bytes (pycmp_typed)
    Doctest: pycmp_typed.test_bytearray_bytes ... ok
    test_bytearray_bytesobject (pycmp_typed)
    Doctest: pycmp_typed.test_bytearray_bytesobject ... ok
    test_bytes_bytearray (pycmp_typed)
    Doctest: pycmp_typed.test_bytes_bytearray ... ok
    test_bytes_bytes (pycmp_typed)
    Doctest: pycmp_typed.test_bytes_bytes ... ok
    test_bytes_bytesobject (pycmp_typed)
    Doctest: pycmp_typed.test_bytes_bytesobject ... ok
    test_bytesobject_bytearray (pycmp_typed)
    Doctest: pycmp_typed.test_bytesobject_bytearray ... ok
    test_bytesobject_bytes (pycmp_typed)
    Doctest: pycmp_typed.test_bytesobject_bytes ... ok
    test_bytesobject_bytesobject (pycmp_typed)
    Doctest: pycmp_typed.test_bytesobject_bytesobject ... ok
    test_float_float (pycmp_typed)
    Doctest: pycmp_typed.test_float_float ... ok
    test_float_int (pycmp_typed)
    Doctest: pycmp_typed.test_float_int ... ok
    test_float_numobject (pycmp_typed)
    Doctest: pycmp_typed.test_float_numobject ... ok
    test_int_float (pycmp_typed)
    Doctest: pycmp_typed.test_int_float ... ok
    test_int_int (pycmp_typed)
    Doctest: pycmp_typed.test_int_int ... FAIL
    test_int_numobject (pycmp_typed)
    Doctest: pycmp_typed.test_int_numobject ... FAIL
    test_numobject_float (pycmp_typed)
    Doctest: pycmp_typed.test_numobject_float ... ok
    test_numobject_int (pycmp_typed)
    Doctest: pycmp_typed.test_numobject_int ... FAIL
    test_numobject_numobject (pycmp_typed)
    Doctest: pycmp_typed.test_numobject_numobject ... FAIL
    test_str_str (pycmp_typed)
    Doctest: pycmp_typed.test_str_str ... ok
    test_str_strobject (pycmp_typed)
    Doctest: pycmp_typed.test_str_strobject ... ok
    test_strobject_str (pycmp_typed)
    Doctest: pycmp_typed.test_strobject_str ... ok
    test_strobject_strobject (pycmp_typed)
    Doctest: pycmp_typed.test_strobject_strobject ... ok
    
    ======================================================================
    FAIL: test_int_int (pycmp_typed)
    Doctest: pycmp_typed.test_int_int
    ----------------------------------------------------------------------
    AssertionError: Failed doctest test for pycmp_typed.test_int_int
      File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18736, in test_int_int
    
    ----------------------------------------------------------------------
    File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18740, in pycmp_typed.test_int_int
    Failed example:
        a = 0; b = 1; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
    Expected:
        (False, True, True, True, False, False)
    Got:
        ERROR: test_int_int(0, 1) ==
        (False, False, True, True, True, True) !=
        (False, True, True, True, False, False)
    ----------------------------------------------------------------------
    File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18742, in pycmp_typed.test_int_int
    Failed example:
        a = 0; b = 2; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
    Expected:
        (False, True, True, True, False, False)
    Got:
        ERROR: test_int_int(0, 2) ==
        (False, False, True, True, True, True) !=
        (False, True, True, True, False, False)
    ----------------------------------------------------------------------
    File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18744, in pycmp_typed.test_int_int
    Failed example:
        a = 0; b = -1; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
    Expected:
        (False, True, False, False, True, True)
    Got:
        ERROR: test_int_int(0, -1) ==
        (False, False, False, False, False, False) !=
        (False, True, False, False, True, True)
    ----------------------------------------------------------------------
    File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18746, in pycmp_typed.test_int_int
    Failed example:
        a = 0; b = -2; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
    Expected:
        (False, True, False, False, True, True)
    Got:
        ERROR: test_int_int(0, -2) ==
        (False, False, False, False, False, False) !=
        (False, True, False, False, True, True)
    ----------------------------------------------------------------------
    File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18748, in pycmp_typed.test_int_int
    Failed example:
        a = 0; b = 32768; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
    Expected:
        (False, True, True, True, False, False)
    Got:
        ERROR: test_int_int(0, 32768) ==
        (False, False, True, True, True, True) !=
        (False, True, True, True, False, False)
    ----------------------------------------------------------------------
    File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18750, in pycmp_typed.test_int_int
    Failed example:
        a = 0; b = 32769; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
    Expected:
        (False, True, True, True, False, False)
    Got:
        ERROR: test_int_int(0, 32769) ==
        (False, False, True, True, True, True) !=
        (False, True, True, True, False, False)
    ----------------------------------------------------------------------
    File "/build/cython/src/cython-3.3.0/TEST_TMP/run/c/pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so", line 18752, in pycmp_typed.test_int_int
    Failed example:
        a = 0; b = -32768; run_test(test_int_int, a, b, (a==b,a!=b,a<b,a<=b,a>=b,a>b))
    Expected:
        (False, True, False, False, True, True)
    Got:
        ERROR: test_int_int(0, -32768) ==
        (False, False, False, False, False, False) !=
        (False, True, False, False, True, True)
    ----------------------------------------------------------------------
    [...] (hundreds of other failures)
    ----------------------------------------------------------------------
    Ran 46 tests in 13.745s
    
    FAILED (failures=8)
    Most expensive pipeline stages: 'parse': 2.74 / 2 (1.370 / run, 54.2%), 'generate_pyx_code_stage': 2.16 / 2 (1.079 / run, 42.6%), 'AnalyseExpressionsTransform': 0.03 / 2 (0.013 / run, 0.5%), 'AnalyseDeclarationsTransform': 0.02 / 2 (0.010 / run, 0.4%), 'ControlFlowAnalysis': 0.01 / 2 (0.005 / run, 0.2%), 'OptimizeBuiltinCalls': 0.01 / 2 (0.004 / run, 0.2%), 'RemoveUnreachableCode': 0.01 / 4 (0.002 / run, 0.2%), 'MarkClosureVisitor': 0.00 / 2 (0.002 / run, 0.1%), 'ConstantFolding': 0.00 / 2 (0.002 / run, 0.1%), 'PostParse': 0.00 / 2 (0.002 / run, 0.1%)
    Times:
    cython      :     5.32 sec  (   2,  2.662 / run) - slowest: 'c:pycmp_typed' (2.91s), 'cpp:pycmp_typed' (2.41s)
    compile-cpp :     3.56 sec  (   1,  3.562 / run) - slowest: 'cpp:pycmp_typed' (3.56s)
    compile-c   :     2.77 sec  (   1,  2.771 / run) - slowest: 'c:pycmp_typed' (2.77s)
    run         :     1.67 sec  (   2,  0.834 / run) - slowest: 'cpp:pycmp_typed' (0.84s), 'c:pycmp_typed' (0.83s)
    import      :     0.00 sec  (   2,  0.000 / run) - slowest: 'c:pycmp_typed' (0.00s), 'cpp:pycmp_typed' (0.00s)
    Test module binary sizes (stripped):
    Total: 9_194_576 bytes / 2 modules = 4_597_288.00 bytes/module
    Smallest modules:    4_595_200 bytes - pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so (4_595_200), pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so (4_599_376)
    Largest modules:     4_599_376 bytes - pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so (4_599_376), pycmp_typed/pycmp_typed.cpython-314-x86_64-linux-gnu.so (4_595_200)
    ALL DONE
    

    Could you also provide the output of the build_env test? It's meant to list relevant information about the test environment for debugging purposes.

    How do I do that? I can't find such test in the cython source.

  4. da-woods commented on Aug 22, 2026

    @da-woods
    Collaborator

    Could you also provide the output of the build_env test? It's meant to list relevant information about the test environment for debugging purposes.

    How do I do that? I can't find such test in the cython source.

    I think it's just python3 runtests.py build_env

  5. antonio-rojas commented on Aug 22, 2026

    @antonio-rojas
    Author

    Running CFLAGS="-O1" python runtests.py pycmp_typed works. So the issue is -O2 (our default build flags)

  6. da-woods commented on Aug 22, 2026

    @da-woods
    Collaborator

    Running CFLAGS="-O1" python runtests.py pycmp_typed works. So the issue is -O2 (our default build flags)

    We run them with -Og I think (mainly because they run much faster like that). I've tried both -O2 and -O3 locally though and failed to reproduce it (although most likely with a different compiler version)

  7. antonio-rojas commented on Aug 22, 2026

    @antonio-rojas
    Author

    A few experiments:

    GCC 16.2.1 and GCC 15.3.0, -O0, -O1, -Og: works
    GCC 16.2.1 and GCC 15.3.0, -O2, -O3: fails
    clang 22.1.8, any -O: works

  8. da-woods commented on Aug 22, 2026

    @da-woods
    Collaborator

    I've had a go with gcc version 16.2.0, gcc version 15.2.1, gcc version 15.3.0 (the opensuse versions, just because that's what's easiest for me to test. I suspect they aren't completely stock) and haven't been able to make them fail I'm afraid.

  9. scoder commented on Aug 23, 2026

    @scoder
    Contributor

    Could you please also provide the output of the build_env test? Just run python3 runtests.py build_env.

    Which C compiler and version are you using? From the looks of this, and the lack of other reports so far (there was a long testing phase for this), we should consider a C compiler bug as a reason. Could you try a different C compiler version?

    EDIT: Sorry, github hadn't updated my ticket view and I hadn't seen your responses.

  10. scoder commented on Aug 23, 2026

    @scoder
    Contributor

    So … maybe even an ArchLinux specific patch that broke gcc for this?

  11. scoder commented on Aug 23, 2026

    @scoder
    Contributor

    I wrote the code that does the comparisons here, so I'm happy to accept bugs on my account, but it's indicative that it's C compiler specific. Apart from a C compiler bug, the second best explanation for optimiser introduced problems in C is usually accidental undefined behaviour, and we have sanitizer checks running that report the test clean, e.g. in https://github.com/cython/cython/actions/runs/32590650966/job/97073860694

  12. antonio-rojas commented on Aug 23, 2026

    @antonio-rojas
    Author

    Could you please also provide the output of the build_env test? Just run python3 runtests.py build_env.

    This doesn't give any more output than the regular python runtests.py. Are you sure that's the right command?

    GNU gdb (GDB) 17.2
    Copyright (C) 2025 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Python 3.14.7 (main, Aug 10 2026, 07:46:56) [GCC 16.1.1 20260728]
    
    Running tests against Cython 3.3.0 bcd04a2966842d80de6ecff4e68e8d892bec5ef7 + uncommitted changes
    Using Cython language level 2.
    Test dependency found: 'numpy' version 2.4.6
    Test dependency not found: 'pythran'
    Test dependency not found: 'setuptools.sandbox'
    Test dependency found: 'asyncio' version 3.14.7
    Test dependency found: 'pstats' version 3.14.7
    Test dependency found: 'posix' version 3.14.7
    Test dependency found: 'array' version 3.14.7
    Test dependency not found: 'Cython.Coverage'
    Test dependency not found: 'Cython.Coverage'
    Test dependency found: 'IPython.testing.globalipapp' version 9.16.1
    Test dependency not found: 'jedi_BROKEN_AND_DISABLED'
    Test dependency not found: 'test.support'
    Backends: c,cpp
    
    ----------------------------------------------------------------------
    Ran 0 tests in 0.000s
    
    NO TESTS RAN
    Most expensive pipeline stages: 
    ALL DONE
    
  13. antonio-rojas commented on Aug 23, 2026

    @antonio-rojas
    Author

    After some more testing (not easy becasue it depends independently on CFLAGS and CXXFLAGS, and they seem to be consumed in several different places), to reproduce this you need -O2 in CFLAGS and -flto=auto in CXXFLAGS:

    CFLAGS="-O2" CXXFLAGS="-flto=auto" python runtests.py pycmp_typed
    

    fails.

    CFLAGS="-O1" CXXFLAGS="-flto=auto" python runtests.py pycmp_typed
    

    works.

    CFLAGS="-O2" CXXFLAGS="" python runtests.py pycmp_typed
    

    also works.

  14. da-woods commented on Aug 23, 2026

    @da-woods
    Collaborator

    -flto=auto reproduces it for me too.

  15. da-woods commented on Aug 23, 2026

    @da-woods
    Collaborator

    #7771 also involved GCC LTO. I'd guess that it might be related (but that's just based on it looking a little similar rather than anything more substantial.

    I'll try to have a look at some of the generate failing code for this one later today though, just to convince myself that there's nothing dodgy-looking there.

  16. scoder commented on Aug 23, 2026

    @scoder
    Contributor

    I'll try to have a look at some of the generate failing code for this one later today though, just to convince myself that there's nothing dodgy-looking there.

    Never hurts to pass a second pair of eyeballs over it. I already did. Some of it feels more dumb than dodgy, but the C compiler should eliminate the redundant parts and make it efficient. I couldn't spot anything suspicious. But I also wrote it, so it might just be author blindness on my side.

  17. antonio-rojas commented on Aug 23, 2026

    @antonio-rojas
    Author

    #7771 also involved GCC LTO. I'd guess that it might be related (but that's just based on it looking a little similar rather than anything more substantial.

    I'll try to have a look at some of the generate failing code for this one later today though, just to convince myself that there's nothing dodgy-looking there.

    Looks like the same issue. -fdump-ipa-icf-details shows

    pycmp_typed.c:20631:18: optimized: Semantic equality hit:__Pyx_PyObject_CompareIntIntEq/8->__Pyx_PyObject_CompareIntIntNe/13
    

    and -fno-ipa-icf makes it work.

  18. da-woods commented on Aug 23, 2026

    @da-woods
    Collaborator

    To me this feels like something we're going to need to fix because it's going to keep coming up (even if it's not our fault).

    I can't see any way of telling the GCC LTO to ignore a specific function (beyond just surrounding it with general "do not optimize" pragmas, which I don't think we want to do).

    I'd imagine the best way might be to make Eq the master version (and the define Ne in terms of Eq) and probably the same thing for </>= and >/<=. That falls apart slight for float and NaN though. But possibly only the int versions are causing problems.

    The other quicker fix would be to turn of USE_LONG_INTERNALS for GCC. That seems to work for me but it's kind of unsatisfactory.

  19. thesamesam commented on Aug 24, 2026

    @thesamesam
    Contributor

    __attribute__((noipa)) should work here if it's an ICF bug, or even __attribute__((optimize("no-ipa-icf"))).

  20. scoder commented on Aug 24, 2026

    @scoder
    Contributor

    I'll rework the code. There is no real reason for separate Python vs. int return variants, so I'll merge those and hopefully fix this issue along the way.

    There is no real reason for separate Python vs. int return variants

    Not entirely true. There is one difference – if we need the PyObject_RichCompare() calllback, then we cannot necessarily go through a C int return value because the result might be an arbitrary Python object, not limited to bool. That's the only reason why I initially wrote it this way.

  21. scoder commented on Aug 24, 2026

    @scoder
    Contributor

    #7919 seems to work.

  22. antonio-rojas commented on Aug 24, 2026

    @antonio-rojas
    Author

    #7919 seems to work.

    Works for me too.

  23. thesamesam commented on Aug 26, 2026

    @thesamesam
    Contributor

    I've not had a chance to try reduce anything here, but if you could try get a minimal-ish C testcase, it'd be much appreciated on the GCC side. I can handle the rest if you could share something standalone.

    It doesn't need to be elegant or make sense, it just needs to be UB-free and exhibit the problem.

  24. scoder commented on Aug 28, 2026

    @scoder
    Contributor

    The two functions that failed to link correctly are these:

    static PyObject* __Pyx_PyObject_CompareIntIntEq(PyObject *op1, PyObject *op2) {
        Py_ssize_t cmp = __Pyx_PyLong_CompareSignAndSize(op1, op2);
        if (cmp == 0) {
            Py_ssize_t size = __Pyx_PyLong_DigitCount(op1);
            if (size > 0) {
                const digit* digits1 = __Pyx_PyLong_Digits(op1);
                const digit* digits2 = __Pyx_PyLong_Digits(op2);
                if (size == 1) {
                    cmp = (Py_ssize_t) digits1[0] - (Py_ssize_t) digits2[0];
                } else if ((size == 2) && (8 * sizeof(Py_ssize_t) >= 2 * PyLong_SHIFT)) {
                    cmp = (Py_ssize_t) (((((size_t)digits1[1]) << PyLong_SHIFT) | (size_t)digits1[0])) - (Py_ssize_t) (((((size_t)digits2[1]) << PyLong_SHIFT) | (size_t)digits2[0]));
                } else {
                    for (Py_ssize_t i=size-1; i >= 0 && !cmp; --i) {
                        cmp = (Py_ssize_t) digits1[i] - (Py_ssize_t) digits2[i];
                    }
                }
            }
            if (cmp == 0) goto __pyx_return_true;
            if (__Pyx_PyLong_IsNeg(op1)) cmp = -cmp;
        }
        goto __pyx_return_false;
    
    __pyx_return_true:
        Py_RETURN_TRUE;
    __pyx_return_false:
        Py_RETURN_FALSE;
    }
    
    static PyObject* __Pyx_PyObject_CompareIntIntNe(PyObject *op1, PyObject *op2) {
        Py_ssize_t cmp = __Pyx_PyLong_CompareSignAndSize(op1, op2);
        if (cmp == 0) {
            Py_ssize_t size = __Pyx_PyLong_DigitCount(op1);
            if (size > 0) {
                const digit* digits1 = __Pyx_PyLong_Digits(op1);
                const digit* digits2 = __Pyx_PyLong_Digits(op2);
                if (size == 1) {
                    cmp = (Py_ssize_t) digits1[0] - (Py_ssize_t) digits2[0];
                } else if ((size == 2) && (8 * sizeof(Py_ssize_t) >= 2 * PyLong_SHIFT)) {
                    cmp = (Py_ssize_t) (((((size_t)digits1[1]) << PyLong_SHIFT) | (size_t)digits1[0])) - (Py_ssize_t) (((((size_t)digits2[1]) << PyLong_SHIFT) | (size_t)digits2[0]));
                } else {
                    for (Py_ssize_t i=size-1; i >= 0 && !cmp; --i) {
                        cmp = (Py_ssize_t) digits1[i] - (Py_ssize_t) digits2[i];
                    }
                }
            }
            if (cmp == 0) goto __pyx_return_false;
            if (__Pyx_PyLong_IsNeg(op1)) cmp = -cmp;
        }
        goto __pyx_return_true;
    
    __pyx_return_true:
        Py_RETURN_TRUE;
    __pyx_return_false:
        Py_RETURN_FALSE;
    }

    The __Pyx_ functions are all macros that do simple bit operations on the fields of two structs that op1 and op2 point to. Most of them should be discardable without changing the failure outcome, although there might have to be some complexity in the functions in order to trigger the bug, can't say. Py_RETURN_TRUE is a macro that increases the reference count to the Py_True singleton and returns it. Py_RETURN_FALSE likewise for Py_False.

    Both functions differ only in two goto statements that have swapped targets in order to return opposite results. The rest is entirely identical.

  25. thesamesam commented on Sep 28, 2026

    @thesamesam
    Contributor

    Thanks. Now handled at #7771 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions