Tarfile regression

[mttpgn]

Bug report

Bug description:

The function _get_filtered_attrs in tarfile.py references the special value os.path.ALLOW_MISSING. This is a problem because this value was only introduced in Python 3.14. However, the reference to this value has been backported to earlier supported Python versions. The result is that tarfile.py can throw an exception when being called from Python versions prior to 3.14.

  File "/usr/local/lib/python3.12/tarfile.py", line 845, in data_filter
    new_attrs = _get_filtered_attrs(member, dest_path, True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/tarfile.py", line 770, in _get_filtered_attrs
    dest_path = os.path.realpath(dest_path, strict=os.path.ALLOW_MISSING)
                                                   ^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'posixpath' has no attribute 'ALLOW_MISSING'

In Python 3.12, the bug first appears in version 3.12.11.

In the Python 3.13 branch, the bug was introduced in 3.13.4, and persists in all subsequent versions.

In Python 3.11, it appears in all versions after and including 3.11.13

And in the Python 3.10 line, 3.10.18 is the first version where this value appears.

Personally, my symptom of this problem is that this bug prevents me from installing common libraries such as numpy because pip uses the tarfile library to unpack the download from PyPI.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[IshaGudewar]

Thanks for reporting this @mttpgn

So The issue is caused by _get_filtered_attrs() in tarfile.py unconditionally referencing
os.path.ALLOW_MISSING, which only exists starting from Python 3.14. This symbol is not
present in Python 3.10–3.13, so calling:

os.path.realpath(dest_path, strict=os.path.ALLOW_MISSING)

raises:

AttributeError: module 'posixpath' has no attribute 'ALLOW_MISSING'

it can reproduce this on Linux with Python 3.12.11, where the error occurs during
`pip install` because pip relies on `tarfile` to extract source distributions
(e.g. numpy, pandas).

This makes package installation fail entirely on affected patch releases:
- 3.10.18+
- 3.11.13+
- 3.12.11+
- 3.13.4+

A simple guard such as:

```python
if hasattr(os.path, "ALLOW_MISSING"):
    dest_path = os.path.realpath(dest_path, strict=os.path.ALLOW_MISSING)
else:
    dest_path = os.path.realpath(dest_path)

would avoid breaking older supported versions.
Until fixed upstream, downgrading to the previous patch version (e.g. 3.12.10)
is the only reliable workaround.

[StanFromIreland]

But it does, it was backported together, e.g. 3.12? You must have some installation issue, maybe you have a stale posixpath.pyc or a broken install where the new tarfile.py was picked up but the old posixpath.py is still being used?

[mttpgn]

Hi Stan from Ireland,

I'm compiling Python from a fresh directory and checking this against the locally compiled result. I tried deleting the pyc file with no luck.

[me@host Python-3.12.12]$ tree -f | grep posixpath*.*pyc
│   │   ├── ./Lib/__pycache__/posixpath.cpython-312.pyc
[me@host Python-3.12.12]$ rm ./Lib/__pycache__/posixpath.cpython-312.pyc 
[me@host Python-3.12.12]$ ./python -c 'import os;print(os.path.ALLOW_MISSING)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'posixpath' has no attribute 'ALLOW_MISSING'

Attempting to install numpy produces the same error afterwards too.

As Isha Gudewar suggested, I do not see this issue at all when attempting the same thing (numpy installation) from a fresh compiled 3.12.10.

[StanFromIreland]

I just pulled the source from python.org, and checked, it all seems correct:

stan@stanlaptop:/tmp/Python-3.12.12$ grep -r "ALLOW_MISSING"
Lib/tarfile.py:    dest_path = os.path.realpath(dest_path, strict=os.path.ALLOW_MISSING)
Lib/tarfile.py:                                   strict=os.path.ALLOW_MISSING)
Lib/tarfile.py:                                           strict=os.path.ALLOW_MISSING)
Lib/posixpath.py:           "commonpath", "isjunction", "ALLOW_MISSING"]
Lib/posixpath.py:    if strict is ALLOW_MISSING:
Lib/genericpath.py:           'samestat', 'ALLOW_MISSING']
Lib/genericpath.py:class ALLOW_MISSING:
Lib/genericpath.py:        return 'os.path.ALLOW_MISSING'

[mttpgn]

Hi Stan,

I agree with you that the string literal "ALLOW_MISSING" is present in the 3.12.12 source code for posixpath.py. However, when I actually execute that code, the variable remains undefined. I'll demonstrate what I mean.

To be sure I am running the right module and not some system posixpath somewhere:

[me@host Python-3.12.12]$ ./python -c 'import os.path; print(os.path.__file__)'
/home/me/bin/Python-3.12.12/Lib/posixpath.py

Yes, that's inside the directory I unpacked from the tarball. Looks right, as it's in a subdirectory of where I compiled this Python from.

That same posixpath.py file contains the string literal in question, as you pointed out:

[me@host Python-3.12.12]$ grep -A7 __all__ /home/me/bin/Python-3.12.12/Lib/posixpath.py
__all__ = ["normcase","isabs","join","splitdrive","splitroot","split","splitext",
           "basename","dirname","commonprefix","getsize","getmtime",
           "getatime","getctime","islink","exists","lexists","isdir","isfile",
           "ismount", "expanduser","expandvars","normpath","abspath",
           "samefile","sameopenfile","samestat",
           "curdir","pardir","sep","pathsep","defpath","altsep","extsep",
           "devnull","realpath","supports_unicode_filenames","relpath",
           "commonpath", "isjunction", "ALLOW_MISSING"]

Toward the top of posixpath, the __all__ variable is overridden with a new list that contains the string in question; however, there's no variable ALLOW_MISSING that's defined anywhere else in the file posixpath.py.

The only other reference to ALLOW_MISSING in posixpath is when its equality is checked against the variable strict; and at that point ALLOW_MISSING is assumed already to exist.

Anyway, when I actually look inside __all__ at runtime, I no longer see that list containing ALLOW_MISSING:

[me@host Python-3.12.12]$ ./python -c 'import os.path; print("ALLOW_MISSING" in os.path.__all__)'
False