Merge branch 'feature/UpdateBackportsHostmatch' into develop

CHANGES.md

@@ -1,5 +1,6 @@
 ### 0.16.0 (2018-xx-xx xx:xx:xx UTC)
 
+* Update backports/ssl_match_hostname 3.5.0.1 (r18) to 3.7.0.1 (r28)
 
 [develop changelog]
 

lib/backports/ssl_match_hostname/LICENSE.txt

@@ -1,51 +0,0 @@
-Python License (Python-2.0)
-
-Python License, Version 2 (Python-2.0)
-
-PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
---------------------------------------------
-
-1. This LICENSE AGREEMENT is between the Python Software Foundation
-("PSF"), and the Individual or Organization ("Licensee") accessing and
-otherwise using this software ("Python") in source or binary form and
-its associated documentation.
-
-2. Subject to the terms and conditions of this License Agreement, PSF
-hereby grants Licensee a nonexclusive, royalty-free, world-wide
-license to reproduce, analyze, test, perform and/or display publicly,
-prepare derivative works, distribute, and otherwise use Python
-alone or in any derivative version, provided, however, that PSF's
-License Agreement and PSF's notice of copyright, i.e., "Copyright (c)
-2001-2013 Python Software Foundation; All Rights Reserved" are retained in
-Python alone or in any derivative version prepared by Licensee.
-
-3. In the event Licensee prepares a derivative work that is based on
-or incorporates Python or any part thereof, and wants to make
-the derivative work available to others as provided herein, then
-Licensee hereby agrees to include in any such work a brief summary of
-the changes made to Python.
-
-4. PSF is making Python available to Licensee on an "AS IS"
-basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
-IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
-DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
-FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
-INFRINGE ANY THIRD PARTY RIGHTS.
-
-5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
-FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
-A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
-OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
-
-6. This License Agreement will automatically terminate upon a material
-breach of its terms and conditions.
-
-7. Nothing in this License Agreement shall be deemed to create any
-relationship of agency, partnership, or joint venture between PSF and
-Licensee. This License Agreement does not grant permission to use PSF
-trademarks or trade name in a trademark sense to endorse or promote
-products or services of Licensee, or any third party.
-
-8. By copying, installing or otherwise using Python, Licensee
-agrees to be bound by the terms and conditions of this License
-Agreement.

lib/backports/ssl_match_hostname/README.txt

@@ -1,69 +0,0 @@
-
-The ssl.match_hostname() function from Python 3.5
-=================================================
-
-The Secure Sockets Layer is only actually *secure*
-if you check the hostname in the certificate returned
-by the server to which you are connecting,
-and verify that it matches to hostname
-that you are trying to reach.
-
-But the matching logic, defined in `RFC2818`_,
-can be a bit tricky to implement on your own.
-So the ``ssl`` package in the Standard Library of Python 3.2
-and greater now includes a ``match_hostname()`` function
-for performing this check instead of requiring every application
-to implement the check separately.
-
-This backport brings ``match_hostname()`` to users
-of earlier versions of Python.
-Simply make this distribution a dependency of your package,
-and then use it like this::
-
-    from backports.ssl_match_hostname import match_hostname, CertificateError
-    [...]
-    sslsock = ssl.wrap_socket(sock, ssl_version=ssl.PROTOCOL_SSLv23,
-                              cert_reqs=ssl.CERT_REQUIRED, ca_certs=...)
-    try:
-        match_hostname(sslsock.getpeercert(), hostname)
-    except CertificateError, ce:
-        ...
-
-Brandon Craig Rhodes is merely the packager of this distribution;
-the actual code inside comes from Python 3.5 with small changes for
-portability.
-
-
-Requirements
-------------
-
-* If you want to verify hosts match with certificates via ServerAltname
-  IPAddress fields, you need to install the `ipaddress module`_.
-  backports.ssl_match_hostname will continue to work without ipaddress but
-  will only be able to handle ServerAltName DNSName fields, not IPAddress.
-  System packagers (Linux distributions, et al) are encouraged to add
-  this as a hard dependency in their packages.
-
-* If you need to use this on Python versions earlier than 2.6 you will need to
-  install the `ssl module`_.  From Python 2.6 upwards ``ssl`` is included in
-  the Python Standard Library so you do not need to install it separately.
-
-.. _`ipaddress module`:: https://pypi.python.org/pypi/ipaddress
-.. _`ssl module`:: https://pypi.python.org/pypi/ssl
-
-History
--------
-
-* This function was introduced in python-3.2
-* It was updated for python-3.4a1 for a CVE 
-  (backports-ssl_match_hostname-3.4.0.1)
-* It was updated from RFC2818 to RFC 6125 compliance in order to fix another
-  security flaw for python-3.3.3 and python-3.4a5
-  (backports-ssl_match_hostname-3.4.0.2)
-* It was updated in python-3.5 to handle IPAddresses in ServerAltName fields
-  (something that backports.ssl_match_hostname will do if you also install the
-  ipaddress library from pypi).
-
-
-.. _RFC2818: http://tools.ietf.org/html/rfc2818.html
-

lib/backports/ssl_match_hostname/__init__.py

@@ -1,82 +1,134 @@
-"""The match_hostname() function from Python 3.3.3, essential when using SSL."""
+"""The match_hostname() function from Python 3.7.0, essential when using SSL."""
 
-import re
 import sys
+import socket as _socket
 
-# ipaddress has been backported to 2.6+ in pypi.  If it is installed on the
-# system, use it to handle IPAddress ServerAltnames (this was added in
-# python-3.5) otherwise only do DNS matching.  This allows
-# backports.ssl_match_hostname to continue to be used all the way back to
-# python-2.4.
 try:
-    import ipaddress
+    # Divergence: Python-3.7+'s _ssl has this exception type but older Pythons do not
-except ImportError:
+    from _ssl import SSLCertVerificationError
-    ipaddress = None
+    CertificateError = SSLCertVerificationError
-
+except:
-__version__ = '3.5.0.1'
+    class CertificateError(ValueError):
+        pass
 
 
-class CertificateError(ValueError):
+__version__ = '3.7.0.1'
-    pass
 
 
-def _dnsname_match(dn, hostname, max_wildcards=1):
+# Divergence: Added to deal with ipaddess as bytes on python2
+def _to_text(obj):
+    if isinstance(obj, str) and sys.version_info < (3,):
+        obj = unicode(obj, encoding='ascii', errors='strict')
+    elif sys.version_info >= (3,) and isinstance(obj, bytes):
+        obj = str(obj, encoding='ascii', errors='strict')
+    return obj
+
+
+def _to_bytes(obj):
+    if isinstance(obj, str) and sys.version_info >= (3,):
+        obj = bytes(obj, encoding='ascii', errors='strict')
+    elif sys.version_info < (3,) and isinstance(obj, unicode):
+        obj = obj.encode('ascii', 'strict')
+    return obj
+
+
+def _dnsname_match(dn, hostname):
     """Matching according to RFC 6125, section 6.4.3
 
-    http://tools.ietf.org/html/rfc6125#section-6.4.3
+    - Hostnames are compared lower case.
+    - For IDNA, both dn and hostname must be encoded as IDN A-label (ACE).
+    - Partial wildcards like 'www*.example.org', multiple wildcards, sole
+      wildcard or wildcards in labels other then the left-most label are not
+      supported and a CertificateError is raised.
+    - A wildcard must match at least one character.
     """
-    pats = []
     if not dn:
         return False
 
-    # Ported from python3-syntax:
+    wildcards = dn.count('*')
-    # leftmost, *remainder = dn.split(r'.')
-    parts = dn.split(r'.')
-    leftmost = parts[0]
-    remainder = parts[1:]
-
-    wildcards = leftmost.count('*')
-    if wildcards > max_wildcards:
-        # Issue #17980: avoid denials of service by refusing more
-        # than one wildcard per fragment.  A survey of established
-        # policy among SSL implementations showed it to be a
-        # reasonable choice.
-        raise CertificateError(
-            "too many wildcards in certificate DNS name: " + repr(dn))
-
     # speed up common case w/o wildcards
     if not wildcards:
         return dn.lower() == hostname.lower()
 
-    # RFC 6125, section 6.4.3, subitem 1.
+    if wildcards > 1:
-    # The client SHOULD NOT attempt to match a presented identifier in which
+        # Divergence .format() to percent formatting for Python < 2.6
-    # the wildcard character comprises a label other than the left-most label.
+        raise CertificateError(
-    if leftmost == '*':
+            "too many wildcards in certificate DNS name: %s" % repr(dn))
-        # When '*' is a fragment by itself, it matches a non-empty dotless
+
-        # fragment.
+    dn_leftmost, sep, dn_remainder = dn.partition('.')
-        pats.append('[^.]+')
+
-    elif leftmost.startswith('xn--') or hostname.startswith('xn--'):
+    if '*' in dn_remainder:
-        # RFC 6125, section 6.4.3, subitem 3.
+        # Only match wildcard in leftmost segment.
-        # The client SHOULD NOT attempt to match a presented identifier
+        # Divergence .format() to percent formatting for Python < 2.6
-        # where the wildcard character is embedded within an A-label or
+        raise CertificateError(
-        # U-label of an internationalized domain name.
+            "wildcard can only be present in the leftmost label: "
-        pats.append(re.escape(leftmost))
+            "%s." % repr(dn))
+
+    if not sep:
+        # no right side
+        # Divergence .format() to percent formatting for Python < 2.6
+        raise CertificateError(
+            "sole wildcard without additional labels are not support: "
+            "%s." % repr(dn))
+
+    if dn_leftmost != '*':
+        # no partial wildcard matching
+        # Divergence .format() to percent formatting for Python < 2.6
+        raise CertificateError(
+            "partial wildcards in leftmost label are not supported: "
+            "%s." % repr(dn))
+
+    hostname_leftmost, sep, hostname_remainder = hostname.partition('.')
+    if not hostname_leftmost or not sep:
+        # wildcard must match at least one char
+        return False
+    return dn_remainder.lower() == hostname_remainder.lower()
+
+
+def _inet_paton(ipname):
+    """Try to convert an IP address to packed binary form
+
+    Supports IPv4 addresses on all platforms and IPv6 on platforms with IPv6
+    support.
+    """
+    # inet_aton() also accepts strings like '1'
+    # Divergence: We make sure we have native string type for all python versions
+    try:
+        b_ipname = _to_bytes(ipname)
+    except UnicodeError:
+        raise ValueError("%s must be an all-ascii string." % repr(ipname))
+
+    # Set ipname in native string format
+    if sys.version_info < (3,):
+        n_ipname = b_ipname
     else:
-        # Otherwise, '*' matches any dotless string, e.g. www*
+        n_ipname = ipname
-        pats.append(re.escape(leftmost).replace(r'\*', '[^.]*'))
 
-    # add the remaining fragments, ignore any wildcards
+    if n_ipname.count('.') == 3:
-    for frag in remainder:
+        try:
-        pats.append(re.escape(frag))
+            return _socket.inet_aton(n_ipname)
+        # Divergence: OSError on late python3.  socket.error earlier.
+        # Null bytes generate ValueError on python3(we want to raise
+        # ValueError anyway), TypeError # earlier
+        except (OSError, _socket.error, TypeError):
+            pass
 
-    pat = re.compile(r'\A' + r'\.'.join(pats) + r'\Z', re.IGNORECASE)
+    try:
-    return pat.match(hostname)
+        return _socket.inet_pton(_socket.AF_INET6, n_ipname)
+    # Divergence: OSError on late python3.  socket.error earlier.
+    # Null bytes generate ValueError on python3(we want to raise
+    # ValueError anyway), TypeError # earlier
+    except (OSError, _socket.error, TypeError):
+        # Divergence .format() to percent formatting for Python < 2.6
+        raise ValueError("%s is neither an IPv4 nor an IP6 "
+                         "address." % repr(ipname))
+    except AttributeError:
+        # AF_INET6 not available
+        pass
 
+    # Divergence .format() to percent formatting for Python < 2.6
+    raise ValueError("%s is not an IPv4 address." % repr(ipname))
 
-def _to_unicode(obj):
-    if isinstance(obj, str) and sys.version_info < (3,):
-        obj = unicode(obj, encoding='ascii', errors='strict')
-    return obj
 
 def _ipaddress_match(ipname, host_ip):
     """Exact matching of IP addresses.
@@ -85,15 +137,19 @@ def _ipaddress_match(ipname, host_ip):
     (section 1.7.2 - "Out of Scope").
     """
     # OpenSSL may add a trailing newline to a subjectAltName's IP address
-    # Divergence from upstream: ipaddress can't handle byte str
+    ip = _inet_paton(ipname.rstrip())
-    ip = ipaddress.ip_address(_to_unicode(ipname).rstrip())
     return ip == host_ip
 
 
 def match_hostname(cert, hostname):
     """Verify that *cert* (in decoded format as returned by
     SSLSocket.getpeercert()) matches the *hostname*.  RFC 2818 and RFC 6125
-    rules are followed, but IP addresses are not accepted for *hostname*.
+    rules are followed.
+
+    The function matches IP addresses rather than dNSNames if hostname is a
+    valid ipaddress string. IPv4 addresses are supported on all platforms.
+    IPv6 addresses are supported on platforms with IPv6 support (AF_INET6
+    and inet_pton).
 
     CertificateError is raised on failure. On success, the function
     returns nothing.
@@ -103,22 +159,16 @@ def match_hostname(cert, hostname):
                          "SSL socket or SSL context with either "
                          "CERT_OPTIONAL or CERT_REQUIRED")
     try:
-        # Divergence from upstream: ipaddress can't handle byte str
+        # Divergence: Deal with hostname as bytes
-        host_ip = ipaddress.ip_address(_to_unicode(hostname))
+        host_ip = _inet_paton(_to_text(hostname))
     except ValueError:
         # Not an IP address (common case)
         host_ip = None
     except UnicodeError:
-        # Divergence from upstream: Have to deal with ipaddress not taking
+        # Divergence: Deal with hostname as byte strings.
-        # byte strings.  addresses should be all ascii, so we consider it not
+        # IP addresses should be all ascii, so we consider it not
-        # an ipaddress in this case
+        # an IP address if this fails
         host_ip = None
-    except AttributeError:
-        # Divergence from upstream: Make ipaddress library optional
-        if ipaddress is None:
-            host_ip = None
-        else:
-            raise
     dnsnames = []
     san = cert.get('subjectAltName', ())
     for key, value in san: