Add os.path.isrelative() and improve ntpath.isabs()

[jongfoster]

BPO	1669539
Nosy	@mhammond, @terryjreedy, @tiran, @tjguk, @ezio-melotti, @bitdancer, @zware, @serhiy-storchaka
Dependencies	bpo-15414: os.path.join behavior on Windows (ntpath.join) is not well documented
Files	win32_absolute_paths.patch: Patch v1 win32_absolute_paths_2.patch: Patch v2 - with docs

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2007-02-26.23:07:51.000>
labels = ['type-feature', 'library']
title = 'Add os.path.isrelative() and improve ntpath.isabs()'
updated_at = <Date 2019-04-26.17:48:31.669>
user = 'https://bugs.python.org/jongfoster'

bugs.python.org fields:

activity = <Date 2019-04-26.17:48:31.669>
actor = 'BreamoreBoy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2007-02-26.23:07:51.000>
creator = 'jongfoster'
dependencies = ['15414']
files = ['7798', '7799']
hgrepos = []
issue_num = 1669539
keywords = ['patch']
message_count = 15.0
messages = ['51978', '51979', '51980', '51981', '51982', '51983', '51984', '58743', '110297', '117582', '138215', '166066', '218560', '222855', '229053']
nosy_count = 11.0
nosy_names = ['mhammond', 'terry.reedy', 'jorend', 'christian.heimes', 'tim.golden', 'ezio.melotti', 'eckhardt', 'r.david.murray', 'zach.ware', 'serhiy.storchaka', 'mdengler']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1669539'
versions = ['Python 3.5']

Linked PRs

• GH-44626, GH-105476: Fix ntpath.isabs() handling of part-absolute paths #113829

[jongfoster]

Hi,

I consider this to be a bug in os.path.isabs():

PythonWin 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32.
>>> import os.path
>>> s = '\\Python25'
>>> os.path.isabs(s)
True
>>> os.path.abspath(s)
'C:\\Python25'
>>> os.chdir('d:')
>>> os.path.abspath(s)
'D:\\Python25'
>>> 

If s is really an absolute path as isabs() claims, then why does abspath() return a different path (i.e. not s)? And worse, note that a call to os.chdir() changes the meaning of s! So s is clearly not an absolute path, and isabs() is wrong.

It turns out that on Windows there are really 4 different kinds of paths:

1. Completely relative, e.g. foo\bar
2. Completely absolute, e.g. c:\foo\bar or \\server\share
3. Halfbreeds with no drive, e.g. \foo\bar
4. Halfbreeds relative to the current working directory on a specific drive, e.g. c:foo\bar

Python 2.5's os.path.isabs() method considers both (2) and (3) to be absolute; I agree with the classification of (2) but strongly disagree about case (3).

Oh, and os.path.join is also broken, e.g. os.path.join('foo', 'a:bar') gives 'foo\\a:bar', which is an invalid path.

Another consequence of this is that os.path.isabs() is not enough to classify paths. Sometimes you really need a relative path, so we really need (at least) a new os.path.isrelative(), which can return "not isabs(s)" on POSIX platforms, and do the right thing on Win32.

The attached patch:

• Changes the behaviour of os.path.isabs() and os.path.join() on Win32, to classify pathnames as described above (including adding UNC support)
• Adds os.path.isrelative() on all platforms
• Changes a couple of Win32 os.path tests where I have deliberately broken backward compatibility
• Adds lots of new tests for these 3 functions on Win32

This does change the behaviour of isabs(), and it is possible that existing applications might be depending on the current behaviour. Silently changing the behaviour might break those applications. I'm not sure what the best course of action is - applying this for 2.6, putting a new API in (os.path.isreallyabs()?), or waiting for Python 3000...?

Kind regards,

Jon Foster

[jongfoster]

P.S. The patch is against Python SVN trunk

[jorend]

Thanks for submitting this. Nice summary of the different cases. I think very few people have given this that much thought. :)

You need a patch for the docs if you want this to even get considered. Then the main obstacle is convincing the team that the nice new logical behavior is worth the breakage. I would be surprised if it got through.

And Python 3000 is not going to be the "subtle breakage" release. I would give you better odds if your patch created new APIs instead, and *documented* exactly what's useful about the model you're suggesting.

[jorend]

There is an interesting thread on python-dev right now about possibly tweaking the behavior of splitext() in 2.6. So maybe there is hope after all.

[jongfoster]

Thanks for the comments. Updated patch attached, which includes the doc changes.
File Added: win32_absolute_paths_2.patch

[mhammond]

I agree the current behaviour is not ideal - but the right way forward isn't clear. Note the existing (strange) behaviour of os.path.join:

>>> os.path.join("c:\\", "\\a")
'c:\\a'
>>> os.path.join("c:\\foo", "\\a")
'\\a'

if isabs('\\a') returns False, there is almost an expectation that the last example should return 'c:\\foo\\a' - but that would almost never give what was expected. 'c:\\a' seems the correct result, but my quick scan of the test code in the patch shows this isn't exercised.

FWIW, about the only common use of isabs() in my code is something like:

log_directory = get_option_from_config_file(...)
if not os.path.isabs(log_directory):
  # make log_directory absolute based on app directory.
  log_directory = os.path.join(app_directory, log_directory)

Best I can tell, the above would not break if isabs('/foo') started returning False, so long as os.path.join('c:\\foo', '\\bar') resulted in 'c:\\bar'.

Off the top-of-my-head, another strategy may be to introduce isrelative() in 2.6, and in 2.7 introduce a warning when isabs() is passed one of the halfbreeds, warning that 2.8 will return a different value, and pointing to the new isrelative(). 2.8 then gets the full, 'correct' implementation as described here.

[jongfoster]

Hi Mark,

Thanks for reviewing this.

You wrote:

os.path.join("c:\\foo", "\\a")
...
'c:\\a' seems the correct result

Yes indeed, and this patch makes it do that. Hiding in the patch is this changed test case:

-tester("ntpath.join('c:/a', '/b')", '/b')
+tester("ntpath.join('c:/a', '/b')", 'c:/b') # CHANGED: Was '/b')

and these new ones:

+tester('ntpath.join("c:\\a", "\\b")', "c:\\b")
+tester('ntpath.join("c:/a", "/b")', "c:/b")

(Hmm looks like one of those tests is a duplicate... not a big deal I guess).

I guess another way to handle these changes is to introduce new isabsolute(), isrelative(), and joinpath() functions, and deprecate isabs() and join(). (By "deprecate" I mean at least put a warning in the docs that it shouldn't be used; whether or not to make the code raise a DeprecationWarning is a separate decision). But that's kinda ugly from an API design point of view. I'm happy to roll a patch that does that, if we decide that fixing isabs() isn't possible.

It's a pity there isn't a way to do something like "from __future__ import fixed_win_paths". But given how dynamic Python is, I guess that would be hard.

Kind regards,

Jon

[tiran]

The problem should be addressed and fixed before the next alpha release.

[BreamoreBoy]

Nosy list changed as both have an interest in Windows issues.

[eckhardt]

I just stumbled across the issue with isabs(). I'd also say that Mark Hammond already provided the typical use case for this, i.e. that you allow relative paths for convenience when storing them or putting them on the commandline, but for actual use you first convert them so that their meaning doesn't change under your feet, e.g. because they depend on volatile things like the current working directory or drive.

If you assume that is the intention or typical use case, then calling isabs() to determine if the path is stable doesn't work. Of course, the wording of the documentation then needs to change, too, as it explicitly says "after chopping off a potential drive letter".

Concerning the behaviour of path.join() and support for "\\server\share" paths, I'm not sure, but I think that these are issues that can be discussed/changed separately.

[terryjreedy]

A bug for tracker purposes is a discrepancy between doc and code. That can be fixed in current versions. A design change is a feature request and can only go in future versions. A deprecation warning for one cycle is desirable when appropriate.

"os.path.isabs(path)
Return True if path is an absolute pathname. On Unix, that means it begins with a slash, on Windows that it begins with a (back)slash after chopping off a potential drive letter."

From what you report, the code does just that, so there is no bug.
Breaking code just because you do not like the current definition is very unlikely to be accepted.

Adding a .isrelative(path) that seems to be equivalent to 'not .ispath' is unlikely to accepted. People who care about the presence of a drive letter can write "path[1]==':'"

"os.path.join(path1[, path2[, ...]])
Join one or more path components intelligently. If any component is an absolute path, all previous components (on Windows, including the previous drive letter, if there was one) are thrown away, and joining continues."

>>> import os
>>> os.path.join('foo', 'a:bar')
'foo\\a:bar'

Producing a non-path does not seem intelligent, so I agree that this a bug. I am not sure what this should produce. 'foo/bar', 'a:foo/bar', and raising an exception would be candidates. All would be better than the current useless return. In any case, a bug fix patch can only fix bugs and should not be intermixed with feature patches.

As to the second sentence: your proposal to change documented and inplemented semantics is, again, a feature request, not a bug fix.

If you want to continue with your feature requests, you can if you wish reclassify this issue, retitle again, and open a new behavior issue for the path bug and upload a bugfix-only patch. Before or along with doing so, I would suggest discussion on the python-ideas list.