Merge branch 'feature.override-asset-with-absolute-path'

Michael Merickel

2014-11-18 92d84360ca61c54008fe81f9943eddf72fd44caf

Merge branch 'feature.override-asset-with-absolute-path'

 CHANGES.txt

@@ -38,6 +38,20 @@
  ``hmac.compare_digest`` if it is available (such as Python 2.7.7+ and 3.3+).
  See https://github.com/Pylons/pyramid/pull/1457

- Assets can now be overidden by an absolute path on the filesystem when using
  the ``config.override_asset`` API. This makes it possible to fully support
  serving up static content from a mutable directory while still being able
  to use the ``request.static_url`` API and ``config.add_static_view``.
  Previously it was not possible to use ``config.add_static_view`` with an
  absolute path **and** generate urls to the content. This change replaces
  the call, ``config.add_static_view('/abs/path', 'static')``, with
  ``config.add_static_view('myapp:static', 'static')`` and
  ``config.override_asset(to_override='myapp:static/',
  override_with='/abs/path/')``. The ``myapp:static`` asset spec is completely
  made up and does not need to exist - it is used for generating urls
  via ``request.static_url('myapp:static/foo.png')``.
  See https://github.com/Pylons/pyramid/issues/1252

Bug Fixes
---------


 docs/narr/assets.rst

@@ -276,15 +276,62 @@
``name`` argument to :meth:`~pyramid.config.Configurator.add_static_view` is
a URL), while keeping static media package-internal and served by the
development webserver during development (if the ``name`` argument to
:meth:`~pyramid.config.Configurator.add_static_view` is a URL prefix).  To
create such a circumstance, we suggest using the
:attr:`pyramid.registry.Registry.settings` API in conjunction with a setting
in the application ``.ini`` file named ``media_location``.  Then set the
value of ``media_location`` to either a prefix or a URL depending on whether
the application is being run in development or in production (use a different
``.ini`` file for production than you do for development).  This is just a
suggestion for a pattern; any setting name other than ``media_location``
could be used.
:meth:`~pyramid.config.Configurator.add_static_view` is a URL prefix).

For example, we may define a :ref:`custom setting <adding_a_custom_setting>`
named ``media_location`` which we can set to an external URL in production
when our assets are hosted on a CDN.

.. code-block:: python
   :linenos:

   media_location = settings.get('media_location', 'static')

   config = Configurator(settings=settings)
   config.add_static_view(path='myapp:static', name=media_location)

Now we can optionally define the setting in our ini file:

.. code-block:: ini
   :linenos:

   # production.ini
   [app:main]
   use = egg:myapp#main

   media_location = http://static.example.com/

It is also possible to serve assets that live outside of the source by
referring to an absolute path on the filesystem. There are two ways to
accomplish this.

First, :meth:`~pyramid.config.Configurator.add_static_view`
supports taking an absolute path directly instead of an asset spec. This works
as expected, looking in the file or folder of files and serving them up at
some URL within your application or externally. Unfortunately, this technique
has a drawback that it is not possible to use the
:meth:`~pyramid.request.Request.static_url` method to generate URLs, since it
works based on an asset spec.

The second approach, available in Pyramid 1.6+, uses the asset overriding
APIs described in the :ref:`overriding_assets_section` section. It is then
possible to configure a "dummy" package which then serves its file or folder
from an absolute path.

.. code-block:: python

   config.add_static_view(path='myapp:static_images', name='static')
   config.override_asset(to_override='myapp:static_images/',
                         override_with='/abs/path/to/images/')

From this configuration it is now possible to use
:meth:`~pyramid.request.Request.static_url` to generate URLs to the data
in the folder by doing something like
``request.static_url('myapp:static_images/foo.png')``. While it is not
necessary that the ``static_images`` file or folder actually exist in the
``myapp`` package, it is important that the ``myapp`` portion points to a
valid package. If the folder does exist then the overriden folder is given
priority if the file's name exists in both locations.

.. index::
   single: Cache Busting
@@ -701,3 +748,6 @@
:func:`pkg_resources.get_resource_string` APIs will obtain an overridden file
when an override is used.

As of Pyramid 1.6, it is also possible to override an asset by supplying an
absolute path to a file or directory. This may be useful if the assets are
not distributed as part of a Python package.

 pyramid/config/assets.py

@@ -1,3 +1,4 @@
import os
import pkg_resources
import sys

@@ -79,7 +80,8 @@
                return result
        return pkg_resources.DefaultProvider.resource_listdir(
            self, resource_name)
        


@implementer(IPackageOverrides)
class PackageOverrides(object):
    # pkg_resources arg in kw args below for testing
@@ -97,57 +99,61 @@
        # optional)...
        # A __loader__ attribute is basically metadata, and setuptools
        # uses it as such.
        package.__loader__ = self 
        package.__loader__ = self
        # we call register_loader_type for every instantiation of this
        # class; that's OK, it's idempotent to do it more than once.
        pkg_resources.register_loader_type(self.__class__, OverrideProvider)
        self.overrides = []
        self.overridden_package_name = package.__name__

    def insert(self, path, package, prefix):
    def insert(self, path, source):
        if not path or path.endswith('/'):
            override = DirectoryOverride(path, package, prefix)
            override = DirectoryOverride(path, source)
        else:
            override = FileOverride(path, package, prefix)
            override = FileOverride(path, source)
        self.overrides.insert(0, override)
        return override

    def search_path(self, resource_name):
    def filtered_sources(self, resource_name):
        for override in self.overrides:
            o = override(resource_name)
            if o is not None:
                package, name = o
                yield package, name
                yield o

    def get_filename(self, resource_name):
        for package, rname in self.search_path(resource_name):
            if pkg_resources.resource_exists(package, rname):
                return pkg_resources.resource_filename(package, rname)
        for source, path in self.filtered_sources(resource_name):
            result = source.get_filename(path)
            if result is not None:
                return result

    def get_stream(self, resource_name):
        for package, rname in self.search_path(resource_name):
            if pkg_resources.resource_exists(package, rname):
                return pkg_resources.resource_stream(package, rname)
        for source, path in self.filtered_sources(resource_name):
            result = source.get_stream(path)
            if result is not None:
                return result

    def get_string(self, resource_name):
        for package, rname in self.search_path(resource_name):
            if pkg_resources.resource_exists(package, rname):
                return pkg_resources.resource_string(package, rname)
        for source, path in self.filtered_sources(resource_name):
            result = source.get_string(path)
            if result is not None:
                return result

    def has_resource(self, resource_name):
        for package, rname in self.search_path(resource_name):
            if pkg_resources.resource_exists(package, rname):
        for source, path in self.filtered_sources(resource_name):
            if source.exists(path):
                return True

    def isdir(self, resource_name):
        for package, rname in self.search_path(resource_name):
            if pkg_resources.resource_exists(package, rname):
                return pkg_resources.resource_isdir(package, rname)
        for source, path in self.filtered_sources(resource_name):
            result = source.isdir(path)
            if result is not None:
                return result

    def listdir(self, resource_name):
        for package, rname in self.search_path(resource_name):
            if pkg_resources.resource_exists(package, rname):
                return pkg_resources.resource_listdir(package, rname)
        for source, path in self.filtered_sources(resource_name):
            result = source.listdir(path)
            if result is not None:
                return result

    @property
    def real_loader(self):
@@ -174,72 +180,180 @@
        """ See IPEP302Loader.
        """
        return self.real_loader.get_source(fullname)
 


class DirectoryOverride:
    def __init__(self, path, package, prefix):
    def __init__(self, path, source):
        self.path = path
        self.package = package
        self.prefix = prefix
        self.pathlen = len(self.path)
        self.source = source

    def __call__(self, resource_name):
        if resource_name.startswith(self.path):
            name = '%s%s' % (self.prefix, resource_name[self.pathlen:])
            return self.package, name
            new_path = resource_name[self.pathlen:]
            return self.source, new_path

class FileOverride:
    def __init__(self, path, package, prefix):
    def __init__(self, path, source):
        self.path = path
        self.package = package
        self.prefix = prefix
        self.source = source

    def __call__(self, resource_name):
        if resource_name == self.path:
            return self.package, self.prefix
            return self.source, ''


class PackageAssetSource(object):
    """
    An asset source relative to a package.

    If this asset source is a file, then we expect the ``prefix`` to point
    to the new name of the file, and the incoming ``resource_name`` will be
    the empty string, as returned by the ``FileOverride``.

    """
    def __init__(self, package, prefix):
        self.package = package
        self.prefix = prefix

    def get_path(self, resource_name):
        return '%s%s' % (self.prefix, resource_name)

    def get_filename(self, resource_name):
        path = self.get_path(resource_name)
        if pkg_resources.resource_exists(self.package, path):
            return pkg_resources.resource_filename(self.package, path)

    def get_stream(self, resource_name):
        path = self.get_path(resource_name)
        if pkg_resources.resource_exists(self.package, path):
            return pkg_resources.resource_stream(self.package, path)

    def get_string(self, resource_name):
        path = self.get_path(resource_name)
        if pkg_resources.resource_exists(self.package, path):
            return pkg_resources.resource_string(self.package, path)

    def exists(self, resource_name):
        path = self.get_path(resource_name)
        if pkg_resources.resource_exists(self.package, path):
            return True

    def isdir(self, resource_name):
        path = self.get_path(resource_name)
        if pkg_resources.resource_exists(self.package, path):
            return pkg_resources.resource_isdir(self.package, path)

    def listdir(self, resource_name):
        path = self.get_path(resource_name)
        if pkg_resources.resource_exists(self.package, path):
            return pkg_resources.resource_listdir(self.package, path)


class FSAssetSource(object):
    """
    An asset source relative to a path in the filesystem.

    """
    def __init__(self, prefix):
        self.prefix = prefix

    def get_filename(self, resource_name):
        if resource_name:
            path = os.path.join(self.prefix, resource_name)
        else:
            path = self.prefix

        if os.path.exists(path):
            return path

    def get_stream(self, resource_name):
        path = self.get_filename(resource_name)
        if path is not None:
            return open(path, 'rb')

    def get_string(self, resource_name):
        stream = self.get_stream(resource_name)
        if stream is not None:
            with stream:
                return stream.read()

    def exists(self, resource_name):
        path = self.get_filename(resource_name)
        if path is not None:
            return True

    def isdir(self, resource_name):
        path = self.get_filename(resource_name)
        if path is not None:
            return os.path.isdir(path)

    def listdir(self, resource_name):
        path = self.get_filename(resource_name)
        if path is not None:
            return os.listdir(path)


class AssetsConfiguratorMixin(object):
    def _override(self, package, path, override_package, override_prefix,
    def _override(self, package, path, override_source,
                  PackageOverrides=PackageOverrides):
        pkg_name = package.__name__
        override_pkg_name = override_package.__name__
        override = self.registry.queryUtility(IPackageOverrides, name=pkg_name)
        if override is None:
            override = PackageOverrides(package)
            self.registry.registerUtility(override, IPackageOverrides,
                                          name=pkg_name)
        override.insert(path, override_pkg_name, override_prefix)
        override.insert(path, override_source)

    @action_method
    def override_asset(self, to_override, override_with, _override=None):
        """ Add a :app:`Pyramid` asset override to the current
        configuration state.

        ``to_override`` is a :term:`asset specification` to the
        ``to_override`` is an :term:`asset specification` to the
        asset being overridden.

        ``override_with`` is a :term:`asset specification` to the
        asset that is performing the override.
        ``override_with`` is an :term:`asset specification` to the
        asset that is performing the override. This may also be an absolute
        path.

        See :ref:`assets_chapter` for more
        information about asset overrides."""
        if to_override == override_with:
            raise ConfigurationError('You cannot override an asset with itself')
            raise ConfigurationError(
                'You cannot override an asset with itself')

        package = to_override
        path = ''
        if ':' in to_override:
            package, path = to_override.split(':', 1)

        override_package = override_with
        override_prefix = ''
        if ':' in override_with:
            override_package, override_prefix = override_with.split(':', 1)

        # *_isdir = override is package or directory
        overridden_isdir = path=='' or path.endswith('/') 
        override_isdir = override_prefix=='' or override_prefix.endswith('/')
        overridden_isdir = path == '' or path.endswith('/')

        if os.path.isabs(override_with):
            override_source = FSAssetSource(override_with)
            if not os.path.exists(override_with):
                raise ConfigurationError(
                    'Cannot override asset with an absolute path that does '
                    'not exist')
            override_isdir = os.path.isdir(override_with)
            override_package = None
            override_prefix = override_with
        else:
            override_package = override_with
            override_prefix = ''
            if ':' in override_with:
                override_package, override_prefix = override_with.split(':', 1)

            __import__(override_package)
            to_package = sys.modules[override_package]
            override_source = PackageAssetSource(to_package, override_prefix)

            override_isdir = (
                override_prefix == '' or
                override_with.endswith('/')
            )

        if overridden_isdir and (not override_isdir):
            raise ConfigurationError(
@@ -255,10 +369,8 @@

        def register():
            __import__(package)
            __import__(override_package)
            from_package = sys.modules[package]
            to_package = sys.modules[override_package]
            override(from_package, path, to_package, override_prefix)
            override(from_package, path, override_source)

        intr = self.introspectable(
            'asset overrides',

 pyramid/tests/test_config/pkgs/asset/subpackage/templates/bar.pt

 pyramid/tests/test_config/test_assets.py

@@ -1,5 +1,9 @@
import os.path
import unittest
from pyramid.testing import cleanUp

# we use this folder
here = os.path.dirname(os.path.abspath(__file__))

class TestAssetsConfiguratorMixin(unittest.TestCase):
    def _makeOne(self, *arg, **kw):
@@ -10,27 +14,31 @@
    def test_override_asset_samename(self):
        from pyramid.exceptions import ConfigurationError
        config = self._makeOne()
        self.assertRaises(ConfigurationError, config.override_asset,'a', 'a')
        self.assertRaises(ConfigurationError, config.override_asset, 'a', 'a')

    def test_override_asset_directory_with_file(self):
        from pyramid.exceptions import ConfigurationError
        config = self._makeOne()
        self.assertRaises(ConfigurationError, config.override_asset,
                          'a:foo/', 'a:foo.pt')
                          'a:foo/',
                          'pyramid.tests.test_config.pkgs.asset:foo.pt')

    def test_override_asset_file_with_directory(self):
        from pyramid.exceptions import ConfigurationError
        config = self._makeOne()
        self.assertRaises(ConfigurationError, config.override_asset,
                          'a:foo.pt', 'a:foo/')
                          'a:foo.pt',
                          'pyramid.tests.test_config.pkgs.asset:templates/')

    def test_override_asset_file_with_package(self):
        from pyramid.exceptions import ConfigurationError
        config = self._makeOne()
        self.assertRaises(ConfigurationError, config.override_asset,
                          'a:foo.pt', 'a')
                          'a:foo.pt',
                          'pyramid.tests.test_config.pkgs.asset')

    def test_override_asset_file_with_file(self):
        from pyramid.config.assets import PackageAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        config.override_asset(
@@ -41,10 +49,13 @@
        from pyramid.tests.test_config.pkgs.asset import subpackage
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, 'templates/foo.pt')
        self.assertEqual(override.override_package, subpackage)
        self.assertEqual(override.override_prefix, 'templates/bar.pt')
        source = override.source
        self.assertTrue(isinstance(source, PackageAssetSource))
        self.assertEqual(source.package, subpackage)
        self.assertEqual(source.prefix, 'templates/bar.pt')

    def test_override_asset_package_with_package(self):
        from pyramid.config.assets import PackageAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        config.override_asset(
@@ -55,10 +66,13 @@
        from pyramid.tests.test_config.pkgs.asset import subpackage
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, '')
        self.assertEqual(override.override_package, subpackage)
        self.assertEqual(override.override_prefix, '')
        source = override.source
        self.assertTrue(isinstance(source, PackageAssetSource))
        self.assertEqual(source.package, subpackage)
        self.assertEqual(source.prefix, '')

    def test_override_asset_directory_with_directory(self):
        from pyramid.config.assets import PackageAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        config.override_asset(
@@ -69,10 +83,13 @@
        from pyramid.tests.test_config.pkgs.asset import subpackage
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, 'templates/')
        self.assertEqual(override.override_package, subpackage)
        self.assertEqual(override.override_prefix, 'templates/')
        source = override.source
        self.assertTrue(isinstance(source, PackageAssetSource))
        self.assertEqual(source.package, subpackage)
        self.assertEqual(source.prefix, 'templates/')

    def test_override_asset_directory_with_package(self):
        from pyramid.config.assets import PackageAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        config.override_asset(
@@ -83,10 +100,13 @@
        from pyramid.tests.test_config.pkgs.asset import subpackage
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, 'templates/')
        self.assertEqual(override.override_package, subpackage)
        self.assertEqual(override.override_prefix, '')
        source = override.source
        self.assertTrue(isinstance(source, PackageAssetSource))
        self.assertEqual(source.package, subpackage)
        self.assertEqual(source.prefix, '')

    def test_override_asset_package_with_directory(self):
        from pyramid.config.assets import PackageAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        config.override_asset(
@@ -97,32 +117,105 @@
        from pyramid.tests.test_config.pkgs.asset import subpackage
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, '')
        self.assertEqual(override.override_package, subpackage)
        self.assertEqual(override.override_prefix, 'templates/')
        source = override.source
        self.assertTrue(isinstance(source, PackageAssetSource))
        self.assertEqual(source.package, subpackage)
        self.assertEqual(source.prefix, 'templates/')

    def test_override_asset_directory_with_absfile(self):
        from pyramid.exceptions import ConfigurationError
        config = self._makeOne()
        self.assertRaises(ConfigurationError, config.override_asset,
                          'a:foo/',
                          os.path.join(here, 'pkgs', 'asset', 'foo.pt'))

    def test_override_asset_file_with_absdirectory(self):
        from pyramid.exceptions import ConfigurationError
        config = self._makeOne()
        abspath = os.path.join(here, 'pkgs', 'asset', 'subpackage', 'templates')
        self.assertRaises(ConfigurationError, config.override_asset,
                          'a:foo.pt',
                          abspath)

    def test_override_asset_file_with_missing_abspath(self):
        from pyramid.exceptions import ConfigurationError
        config = self._makeOne()
        self.assertRaises(ConfigurationError, config.override_asset,
                          'a:foo.pt',
                          os.path.join(here, 'wont_exist'))

    def test_override_asset_file_with_absfile(self):
        from pyramid.config.assets import FSAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        abspath = os.path.join(here, 'pkgs', 'asset', 'subpackage',
                               'templates', 'bar.pt')
        config.override_asset(
            'pyramid.tests.test_config.pkgs.asset:templates/foo.pt',
            abspath,
            _override=override)
        from pyramid.tests.test_config.pkgs import asset
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, 'templates/foo.pt')
        source = override.source
        self.assertTrue(isinstance(source, FSAssetSource))
        self.assertEqual(source.prefix, abspath)

    def test_override_asset_directory_with_absdirectory(self):
        from pyramid.config.assets import FSAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        abspath = os.path.join(here, 'pkgs', 'asset', 'subpackage', 'templates')
        config.override_asset(
            'pyramid.tests.test_config.pkgs.asset:templates/',
            abspath,
            _override=override)
        from pyramid.tests.test_config.pkgs import asset
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, 'templates/')
        source = override.source
        self.assertTrue(isinstance(source, FSAssetSource))
        self.assertEqual(source.prefix, abspath)

    def test_override_asset_package_with_absdirectory(self):
        from pyramid.config.assets import FSAssetSource
        config = self._makeOne(autocommit=True)
        override = DummyUnderOverride()
        abspath = os.path.join(here, 'pkgs', 'asset', 'subpackage', 'templates')
        config.override_asset(
            'pyramid.tests.test_config.pkgs.asset',
            abspath,
            _override=override)
        from pyramid.tests.test_config.pkgs import asset
        self.assertEqual(override.package, asset)
        self.assertEqual(override.path, '')
        source = override.source
        self.assertTrue(isinstance(source, FSAssetSource))
        self.assertEqual(source.prefix, abspath)

    def test__override_not_yet_registered(self):
        from pyramid.interfaces import IPackageOverrides
        package = DummyPackage('package')
        opackage = DummyPackage('opackage')
        source = DummyAssetSource()
        config = self._makeOne()
        config._override(package, 'path', opackage, 'oprefix',
        config._override(package, 'path', source,
                         PackageOverrides=DummyPackageOverrides)
        overrides = config.registry.queryUtility(IPackageOverrides,
                                                 name='package')
        self.assertEqual(overrides.inserted, [('path', 'opackage', 'oprefix')])
        self.assertEqual(overrides.inserted, [('path', source)])
        self.assertEqual(overrides.package, package)

    def test__override_already_registered(self):
        from pyramid.interfaces import IPackageOverrides
        package = DummyPackage('package')
        opackage = DummyPackage('opackage')
        source = DummyAssetSource()
        overrides = DummyPackageOverrides(package)
        config = self._makeOne()
        config.registry.registerUtility(overrides, IPackageOverrides,
                                        name='package')
        config._override(package, 'path', opackage, 'oprefix',
        config._override(package, 'path', source,
                         PackageOverrides=DummyPackageOverrides)
        self.assertEqual(overrides.inserted, [('path', 'opackage', 'oprefix')])
        self.assertEqual(overrides.inserted, [('path', source)])
        self.assertEqual(overrides.package, package)


@@ -148,30 +241,24 @@
        reg.registerUtility(overrides, IPackageOverrides, name=name)

    def test_get_resource_filename_no_overrides(self):
        import os
        resource_name = 'test_assets.py'
        import pyramid.tests.test_config
        provider = self._makeOne(pyramid.tests.test_config)
        here = os.path.dirname(os.path.abspath(__file__))
        expected = os.path.join(here, resource_name)
        result = provider.get_resource_filename(None, resource_name)
        self.assertEqual(result, expected)

    def test_get_resource_stream_no_overrides(self):
        import os
        resource_name = 'test_assets.py'
        import pyramid.tests.test_config
        provider = self._makeOne(pyramid.tests.test_config)
        here = os.path.dirname(os.path.abspath(__file__))
        with provider.get_resource_stream(None, resource_name) as result:
            _assertBody(result.read(), os.path.join(here, resource_name))

    def test_get_resource_string_no_overrides(self):
        import os
        resource_name = 'test_assets.py'
        import pyramid.tests.test_config
        provider = self._makeOne(pyramid.tests.test_config)
        here = os.path.dirname(os.path.abspath(__file__))
        result = provider.get_resource_string(None, resource_name)
        _assertBody(result, os.path.join(here, resource_name))

@@ -202,11 +289,9 @@
    def test_get_resource_filename_override_returns_None(self):
        overrides = DummyOverrides(None)
        self._registerOverrides(overrides)
        import os
        resource_name = 'test_assets.py'
        import pyramid.tests.test_config
        provider = self._makeOne(pyramid.tests.test_config)
        here = os.path.dirname(os.path.abspath(__file__))
        expected = os.path.join(here, resource_name)
        result = provider.get_resource_filename(None, resource_name)
        self.assertEqual(result, expected)
@@ -214,22 +299,18 @@
    def test_get_resource_stream_override_returns_None(self):
        overrides = DummyOverrides(None)
        self._registerOverrides(overrides)
        import os
        resource_name = 'test_assets.py'
        import pyramid.tests.test_config
        provider = self._makeOne(pyramid.tests.test_config)
        here = os.path.dirname(os.path.abspath(__file__))
        with provider.get_resource_stream(None, resource_name) as result:
            _assertBody(result.read(), os.path.join(here, resource_name))

    def test_get_resource_string_override_returns_None(self):
        overrides = DummyOverrides(None)
        self._registerOverrides(overrides)
        import os
        resource_name = 'test_assets.py'
        import pyramid.tests.test_config
        provider = self._makeOne(pyramid.tests.test_config)
        here = os.path.dirname(os.path.abspath(__file__))
        result = provider.get_resource_string(None, resource_name)
        _assertBody(result, os.path.join(here, resource_name))

@@ -378,8 +459,8 @@
        from pyramid.config.assets import DirectoryOverride
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= [None]
        po.insert('foo/', 'package', 'bar/')
        po.overrides = [None]
        po.insert('foo/', DummyAssetSource())
        self.assertEqual(len(po.overrides), 2)
        override = po.overrides[0]
        self.assertEqual(override.__class__, DirectoryOverride)
@@ -388,8 +469,8 @@
        from pyramid.config.assets import FileOverride
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= [None]
        po.insert('foo.pt', 'package', 'bar.pt')
        po.overrides = [None]
        po.insert('foo.pt', DummyAssetSource())
        self.assertEqual(len(po.overrides), 2)
        override = po.overrides[0]
        self.assertEqual(override.__class__, FileOverride)
@@ -399,132 +480,137 @@
        from pyramid.config.assets import DirectoryOverride
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= [None]
        po.insert('', 'package', 'bar/')
        po.overrides = [None]
        source = DummyAssetSource()
        po.insert('', source)
        self.assertEqual(len(po.overrides), 2)
        override = po.overrides[0]
        self.assertEqual(override.__class__, DirectoryOverride)

    def test_search_path(self):
        overrides = [ DummyOverride(None), DummyOverride(('package', 'name'))]
    def test_filtered_sources(self):
        overrides = [ DummyOverride(None), DummyOverride('foo')]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        self.assertEqual(list(po.search_path('whatever')),
                         [('package', 'name')])
        po.overrides = overrides
        self.assertEqual(list(po.filtered_sources('whatever')), ['foo'])

    def test_get_filename(self):
        import os
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'test_assets.py'))]
        source = DummyAssetSource(filename='foo.pt')
        overrides = [ DummyOverride(None), DummyOverride((source, ''))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        here = os.path.dirname(os.path.abspath(__file__))
        expected = os.path.join(here, 'test_assets.py')
        self.assertEqual(po.get_filename('whatever'), expected)
        po.overrides = overrides
        result = po.get_filename('whatever')
        self.assertEqual(result, 'foo.pt')
        self.assertEqual(source.resource_name, '')

    def test_get_filename_file_doesnt_exist(self):
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'wont_exist'))]
        source = DummyAssetSource(filename=None)
        overrides = [DummyOverride(None), DummyOverride((source, 'wont_exist'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.get_filename('whatever'), None)
        
        self.assertEqual(source.resource_name, 'wont_exist')

    def test_get_stream(self):
        import os
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'test_assets.py'))]
        source = DummyAssetSource(stream='a stream?')
        overrides = [DummyOverride(None), DummyOverride((source, 'foo.pt'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        here = os.path.dirname(os.path.abspath(__file__))
        with po.get_stream('whatever') as stream:
            _assertBody(stream.read(), os.path.join(here, 'test_assets.py'))
        po.overrides = overrides
        self.assertEqual(po.get_stream('whatever'), 'a stream?')
        self.assertEqual(source.resource_name, 'foo.pt')
        
    def test_get_stream_file_doesnt_exist(self):
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'wont_exist'))]
        source = DummyAssetSource(stream=None)
        overrides = [DummyOverride(None), DummyOverride((source, 'wont_exist'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.get_stream('whatever'), None)
        self.assertEqual(source.resource_name, 'wont_exist')

    def test_get_string(self):
        import os
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'test_assets.py'))]
        source = DummyAssetSource(string='a string')
        overrides = [DummyOverride(None), DummyOverride((source, 'foo.pt'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        here = os.path.dirname(os.path.abspath(__file__))
        _assertBody(po.get_string('whatever'),
                    os.path.join(here, 'test_assets.py'))
        po.overrides = overrides
        self.assertEqual(po.get_string('whatever'), 'a string')
        self.assertEqual(source.resource_name, 'foo.pt')
        
    def test_get_string_file_doesnt_exist(self):
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'wont_exist'))]
        source = DummyAssetSource(string=None)
        overrides = [DummyOverride(None), DummyOverride((source, 'wont_exist'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.get_string('whatever'), None)
        self.assertEqual(source.resource_name, 'wont_exist')

    def test_has_resource(self):
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'test_assets.py'))]
        source = DummyAssetSource(exists=True)
        overrides = [DummyOverride(None), DummyOverride((source, 'foo.pt'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.has_resource('whatever'), True)
        self.assertEqual(source.resource_name, 'foo.pt')

    def test_has_resource_file_doesnt_exist(self):
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'wont_exist'))]
        source = DummyAssetSource(exists=None)
        overrides = [DummyOverride(None), DummyOverride((source, 'wont_exist'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.has_resource('whatever'), None)
        self.assertEqual(source.resource_name, 'wont_exist')

    def test_isdir_false(self):
        overrides = [ DummyOverride(
            ('pyramid.tests.test_config', 'test_assets.py'))]
        source = DummyAssetSource(isdir=False)
        overrides = [DummyOverride(None), DummyOverride((source, 'foo.pt'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.isdir('whatever'), False)
        
        self.assertEqual(source.resource_name, 'foo.pt')

    def test_isdir_true(self):
        overrides = [ DummyOverride(
            ('pyramid.tests.test_config', 'files'))]
        source = DummyAssetSource(isdir=True)
        overrides = [DummyOverride(None), DummyOverride((source, 'foo.pt'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.isdir('whatever'), True)
        self.assertEqual(source.resource_name, 'foo.pt')

    def test_isdir_doesnt_exist(self):
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'wont_exist'))]
        source = DummyAssetSource(isdir=None)
        overrides = [DummyOverride(None), DummyOverride((source, 'wont_exist'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.isdir('whatever'), None)
        self.assertEqual(source.resource_name, 'wont_exist')

    def test_listdir(self):
        overrides = [ DummyOverride(
            ('pyramid.tests.test_config', 'files'))]
        source = DummyAssetSource(listdir=True)
        overrides = [DummyOverride(None), DummyOverride((source, 'foo.pt'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        self.assertTrue(po.listdir('whatever'))
        po.overrides = overrides
        self.assertEqual(po.listdir('whatever'), True)
        self.assertEqual(source.resource_name, 'foo.pt')

    def test_listdir_doesnt_exist(self):
        overrides = [ DummyOverride(None), DummyOverride(
            ('pyramid.tests.test_config', 'wont_exist'))]
        source = DummyAssetSource(listdir=None)
        overrides = [DummyOverride(None), DummyOverride((source, 'wont_exist'))]
        package = DummyPackage('package')
        po = self._makeOne(package)
        po.overrides= overrides
        po.overrides = overrides
        self.assertEqual(po.listdir('whatever'), None)
        self.assertEqual(source.resource_name, 'wont_exist')

    # PEP 302 __loader__ extensions:  use the "real" __loader__, if present.
    def test_get_data_pkg_has_no___loader__(self):
@@ -570,27 +656,124 @@

    def test_get_source_pkg_has___loader__(self):
        package = DummyPackage('package')
        loader = package.__loader__  = DummyLoader()
        loader = package.__loader__ = DummyLoader()
        po = self._makeOne(package)
        self.assertEqual(po.get_source('whatever'), 'def foo():\n    pass')
        self.assertEqual(loader._got_source, 'whatever')

class AssetSourceIntegrationTests(object):

    def test_get_filename(self):
        source = self._makeOne('')
        self.assertEqual(source.get_filename('test_assets.py'),
                         os.path.join(here, 'test_assets.py'))

    def test_get_filename_with_prefix(self):
        source = self._makeOne('test_assets.py')
        self.assertEqual(source.get_filename(''),
                         os.path.join(here, 'test_assets.py'))

    def test_get_filename_file_doesnt_exist(self):
        source = self._makeOne('')
        self.assertEqual(source.get_filename('wont_exist'), None)

    def test_get_stream(self):
        source = self._makeOne('')
        with source.get_stream('test_assets.py') as stream:
            _assertBody(stream.read(), os.path.join(here, 'test_assets.py'))

    def test_get_stream_with_prefix(self):
        source = self._makeOne('test_assets.py')
        with source.get_stream('') as stream:
            _assertBody(stream.read(), os.path.join(here, 'test_assets.py'))

    def test_get_stream_file_doesnt_exist(self):
        source = self._makeOne('')
        self.assertEqual(source.get_stream('wont_exist'), None)

    def test_get_string(self):
        source = self._makeOne('')
        _assertBody(source.get_string('test_assets.py'),
                    os.path.join(here, 'test_assets.py'))

    def test_get_string_with_prefix(self):
        source = self._makeOne('test_assets.py')
        _assertBody(source.get_string(''),
                    os.path.join(here, 'test_assets.py'))

    def test_get_string_file_doesnt_exist(self):
        source = self._makeOne('')
        self.assertEqual(source.get_string('wont_exist'), None)

    def test_exists(self):
        source = self._makeOne('')
        self.assertEqual(source.exists('test_assets.py'), True)

    def test_exists_with_prefix(self):
        source = self._makeOne('test_assets.py')
        self.assertEqual(source.exists(''), True)

    def test_exists_file_doesnt_exist(self):
        source = self._makeOne('')
        self.assertEqual(source.exists('wont_exist'), None)

    def test_isdir_false(self):
        source = self._makeOne('')
        self.assertEqual(source.isdir('test_assets.py'), False)

    def test_isdir_true(self):
        source = self._makeOne('')
        self.assertEqual(source.isdir('files'), True)

    def test_isdir_doesnt_exist(self):
        source = self._makeOne('')
        self.assertEqual(source.isdir('wont_exist'), None)

    def test_listdir(self):
        source = self._makeOne('')
        self.assertTrue(source.listdir('files'))

    def test_listdir_doesnt_exist(self):
        source = self._makeOne('')
        self.assertEqual(source.listdir('wont_exist'), None)

class TestPackageAssetSource(AssetSourceIntegrationTests, unittest.TestCase):

    def _getTargetClass(self):
        from pyramid.config.assets import PackageAssetSource
        return PackageAssetSource

    def _makeOne(self, prefix, package='pyramid.tests.test_config'):
        klass = self._getTargetClass()
        return klass(package, prefix)

class TestFSAssetSource(AssetSourceIntegrationTests, unittest.TestCase):
    def _getTargetClass(self):
        from pyramid.config.assets import FSAssetSource
        return FSAssetSource

    def _makeOne(self, prefix, base_prefix=here):
        klass = self._getTargetClass()
        return klass(os.path.join(base_prefix, prefix))

class TestDirectoryOverride(unittest.TestCase):
    def _getTargetClass(self):
        from pyramid.config.assets import DirectoryOverride
        return DirectoryOverride

    def _makeOne(self, path, package, prefix):
    def _makeOne(self, path, source):
        klass = self._getTargetClass()
        return klass(path, package, prefix)
        return klass(path, source)

    def test_it_match(self):
        o = self._makeOne('foo/', 'package', 'bar/')
        source = DummyAssetSource()
        o = self._makeOne('foo/', source)
        result = o('foo/something.pt')
        self.assertEqual(result, ('package', 'bar/something.pt'))
        self.assertEqual(result, (source, 'something.pt'))
        
    def test_it_no_match(self):
        o = self._makeOne('foo/', 'package', 'bar/')
        source = DummyAssetSource()
        o = self._makeOne('foo/', source)
        result = o('baz/notfound.pt')
        self.assertEqual(result, None)

@@ -599,17 +782,19 @@
        from pyramid.config.assets import FileOverride
        return FileOverride

    def _makeOne(self, path, package, prefix):
    def _makeOne(self, path, source):
        klass = self._getTargetClass()
        return klass(path, package, prefix)
        return klass(path, source)

    def test_it_match(self):
        o = self._makeOne('foo.pt', 'package', 'bar.pt')
        source = DummyAssetSource()
        o = self._makeOne('foo.pt', source)
        result = o('foo.pt')
        self.assertEqual(result, ('package', 'bar.pt'))
        self.assertEqual(result, (source, ''))
        
    def test_it_no_match(self):
        o = self._makeOne('foo.pt', 'package', 'bar.pt')
        source = DummyAssetSource()
        o = self._makeOne('foo.pt', source)
        result = o('notfound.pt')
        self.assertEqual(result, None)

@@ -634,8 +819,8 @@
        self.package = package
        self.inserted = []

    def insert(self, path, package, prefix):
        self.inserted.append((path, package, prefix))
    def insert(self, path, source):
        self.inserted.append((path, source))
    
class DummyPkgResources:
    def __init__(self):
@@ -647,6 +832,34 @@
class DummyPackage:
    def __init__(self, name):
        self.__name__ = name

class DummyAssetSource:
    def __init__(self, **kw):
        self.kw = kw

    def get_filename(self, resource_name):
        self.resource_name = resource_name
        return self.kw['filename']

    def get_stream(self, resource_name):
        self.resource_name = resource_name
        return self.kw['stream']

    def get_string(self, resource_name):
        self.resource_name = resource_name
        return self.kw['string']

    def exists(self, resource_name):
        self.resource_name = resource_name
        return self.kw['exists']

    def isdir(self, resource_name):
        self.resource_name = resource_name
        return self.kw['isdir']

    def listdir(self, resource_name):
        self.resource_name = resource_name
        return self.kw['listdir']
 
class DummyLoader:
    _got_data = _is_package = None
@@ -664,12 +877,10 @@
        return 'def foo():\n    pass'

class DummyUnderOverride:
    def __call__(self, package, path, override_package, override_prefix,
                 _info=''):
    def __call__(self, package, path, source, _info=''):
        self.package = package
        self.path = path
        self.override_package = override_package
        self.override_prefix = override_prefix
        self.source = source

def read_(src):
    with open(src, 'rb') as f: