RedHatTraining/pyramid.git

fix the docs for route_prefix_context to avoid showing up as a doctest

Michael Merickel

2018-10-17 e14661417e7ceb50d5cf83bbd6abd6b133e473ba

commit \| author \| age
672bbe	1	.. _design_defense:
CM	2
edd915	3	Defending Pyramid's Design
CM	4	==========================
fbfea7	5
8cdb1b	6	From time to time, challenges to various aspects of :app:`Pyramid` design are
CM	7	lodged. To give context to discussions that follow, we detail some of the
	8	design decisions and trade-offs here. In some cases, we acknowledge that the
	9	framework can be made better and we describe future steps which will be taken
1dea14	10	to improve it. In others we just file the challenge as noted, as obviously you
SP	11	can't please everyone all of the time.
70f7c3	12
6cdbba	13	Pyramid Provides More Than One Way to Do It
CM	14	-------------------------------------------
	15
dfc9d6	16	A canon of Python popular culture is "TIOOWTDI" ("there is only one way to do
1dea14	17	it", a slighting, tongue-in-cheek reference to Perl's "TIMTOWTDI", which is an
SP	18	acronym for "there is more than one way to do it").
6cdbba	19
1dea14	20	:app:`Pyramid` is, for better or worse, a "TIMTOWTDI" system. For example, it
SP	21	includes more than one way to resolve a URL to a :term:`view callable`: via
	22	:term:`url dispatch` or :term:`traversal`. Multiple methods of configuration
	23	exist: :term:`imperative configuration`, :term:`configuration decoration`, and
	24	:term:`ZCML` (optionally via :term:`pyramid_zcml`). It works with multiple
	25	different kinds of persistence and templating systems. And so on. However, the
	26	existence of most of these overlapping ways to do things are not without reason
	27	and purpose: we have a number of audiences to serve, and we believe that
	28	TIMTOWTDI at the web framework level actually prevents a much more insidious
	29	and harmful set of duplication at higher levels in the Python web community.
6cdbba	30
1dea14	31	:app:`Pyramid` began its life as :mod:`repoze.bfg`, written by a team of people
SP	32	with many years of prior :term:`Zope` experience. The idea of
bee624	33	:term:`traversal` and the way :term:`view lookup` works was stolen entirely
CM	34	from Zope. The authorization subsystem provided by :app:`Pyramid` is a
	35	derivative of Zope's. The idea that an application can be extended without
	36	forking is also a Zope derivative.
6cdbba	37
CM	38	Implementations of these features were required to allow the :app:`Pyramid`
1dea14	39	authors to build the bread-and-butter CMS-type systems for customers in the way
SP	40	in which they were accustomed. No other system, save for Zope itself, had such
	41	features, and Zope itself was beginning to show signs of its age. We were
	42	becoming hampered by consequences of its early design mistakes. Zope's lack of
	43	documentation was also difficult to work around. It was hard to hire smart
	44	people to work on Zope applications because there was no comprehensive
	45	documentation set which explained "it all" in one consumable place, and it was
	46	too large and self-inconsistent to document properly. Before :mod:`repoze.bfg`
	47	went under development, its authors obviously looked around for other
	48	frameworks that fit the bill. But no non-Zope framework did. So we embarked on
	49	building :mod:`repoze.bfg`.
6cdbba	50
CM	51	As the result of our research, however, it became apparent that, despite the
1dea14	52	fact that no one framework had all the features we required, lots of existing
SP	53	frameworks had good, and sometimes very compelling ideas. In particular,
	54	:term:`URL dispatch` is a more direct mechanism to map URLs to code.
6cdbba	55
2e54b8	56	So, although we couldn't find a framework, save for Zope, that fit our needs,
6cdbba	57	and while we incorporated a lot of Zope ideas into BFG, we also emulated the
CM	58	features we found compelling in other frameworks (such as :term:`url
1dea14	59	dispatch`). After the initial public release of BFG, as time went on, features
SP	60	were added to support people allergic to various Zope-isms in the system, such
	61	as the ability to configure the application using :term:`imperative
	62	configuration` and :term:`configuration decoration`, rather than solely using
	63	:term:`ZCML`, and the elimination of the required use of :term:`interface`
	64	objects. It soon became clear that we had a system that was very generic, and
	65	was beginning to appeal to non-Zope users as well as ex-Zope users.
6cdbba	66
CM	67	As the result of this generalization, it became obvious BFG shared 90% of its
1dea14	68	feature set with the feature set of Pylons 1, and thus had a very similar
SP	69	target market. Because they were so similar, choosing between the two systems
	70	was an exercise in frustration for an otherwise non-partisan developer. It was
	71	also strange for the Pylons and BFG development communities to be in
	72	competition for the same set of users, given how similar the two frameworks
	73	were. So the Pylons and BFG teams began to work together to form a plan to
	74	merge. The features missing from BFG (notably :term:`view handler` classes,
	75	flash messaging, and other minor missing bits), were added to provide
	76	familiarity to ex-Pylons users. The result is :app:`Pyramid`.
6cdbba	77
1dea14	78	The Python web framework space is currently notoriously balkanized. We're truly
SP	79	hoping that the amalgamation of components in :app:`Pyramid` will appeal to at
	80	least two currently very distinct sets of users: Pylons and BFG users. By
	81	unifying the best concepts from Pylons and BFG into a single codebase, and
	82	leaving the bad concepts from their ancestors behind, we'll be able to
	83	consolidate our efforts better, share more code, and promote our efforts as a
	84	unit rather than competing pointlessly. We hope to be able to shortcut the pack
	85	mentality which results in a much larger duplication of effort, represented
	86	by competing but incredibly similar applications and libraries, each built upon
	87	a specific low level stack that is incompatible with the other. We'll also
	88	shrink the choice of credible Python web frameworks down by at least one. We're
	89	also hoping to attract users from other communities (such as Zope's and
	90	TurboGears') by providing the features they require, while allowing enough
	91	flexibility to do things in a familiar fashion. Some overlap of functionality
	92	to achieve these goals is expected and unavoidable, at least if we aim to
	93	prevent pointless duplication at higher levels. If we've done our job well
	94	enough, the various audiences will be able to coexist and cooperate rather than
	95	firing at each other across some imaginary web framework DMZ.
6cdbba	96
1dea14	97	Pyramid Uses a Zope Component Architecture ("ZCA") Registry
edd915	98	-----------------------------------------------------------
fbfea7	99
8cdb1b	100	:app:`Pyramid` uses a :term:`Zope Component Architecture` (ZCA) "component
CM	101	registry" as its :term:`application registry` under the hood. This is a
	102	point of some contention. :app:`Pyramid` is of a :term:`Zope` pedigree, so
	103	it was natural for its developers to use a ZCA registry at its inception.
	104	However, we understand that using a ZCA registry has issues and consequences,
	105	which we've attempted to address as best we can. Here's an introspection
	106	about :app:`Pyramid` use of a ZCA registry, and the trade-offs its usage
0435ce	107	involves.
fbfea7	108
CM	109	Problems
	110	++++++++
	111
96d456	112	The global API that may be used to access data in a ZCA component registry
CM	113	is not particularly pretty or intuitive, and sometimes it's just plain
	114	obtuse. Likewise, the conceptual load on a casual source code reader of code
	115	that uses the ZCA global API is somewhat high. Consider a ZCA neophyte
	116	reading the code that performs a typical "unnamed utility" lookup using the
	117	:func:`zope.component.getUtility` global API:
fbfea7	118
CM	119	.. code-block:: python
4ef406	120	:linenos:
fbfea7	121
4ef406	122	from pyramid.interfaces import ISettings
SP	123	from zope.component import getUtility
	124	settings = getUtility(ISettings)
fbfea7	125
8cdb1b	126	After this code runs, ``settings`` will be a Python dictionary. But it's
96d456	127	unlikely that any civilian would know that just by reading the code. There
8cdb1b	128	are a number of comprehension issues with the bit of code above that are
CM	129	obvious.
fbfea7	130
8cdb1b	131	First, what's a "utility"? Well, for the purposes of this discussion, and
CM	132	for the purpose of the code above, it's just not very important. If you
	133	really want to know, you can read `this
1cb30e	134	<http://muthukadan.net/docs/zca.html#utility>`_. However, still, readers
8cdb1b	135	of such code need to understand the concept in order to parse it. This is
CM	136	problem number one.
fbfea7	137
8cdb1b	138	Second, what's this ``ISettings`` thing? It's an :term:`interface`. Is that
96d456	139	important here? Not really, we're just using it as a key for some lookup
8cdb1b	140	based on its identity as a marker: it represents an object that has the
CM	141	dictionary API, but that's not very important in this context. That's
	142	problem number two.
fbfea7	143
8cdb1b	144	Third of all, what does the ``getUtility`` function do? It's performing a
e8609b	145	lookup for the ``ISettings`` "utility" that should return... well, a utility.
8cdb1b	146	Note how we've already built up a dependency on the understanding of an
CM	147	:term:`interface` and the concept of "utility" to answer this question: a bad
	148	sign so far. Note also that the answer is circular, a really bad sign.
fbfea7	149
8cdb1b	150	Fourth, where does ``getUtility`` look to get the data? Well, the "component
CM	151	registry" of course. What's a component registry? Problem number four.
fbfea7	152
8cdb1b	153	Fifth, assuming you buy that there's some magical registry hanging around,
CM	154	where is this registry? Homina homina... "around"? That's sort of the
e8609b	155	best answer in this context (a more specific answer would require knowledge of
SP	156	internals). Can there be more than one registry? Yes. So in which registry
	157	does it find the registration? Well, the "current" registry of course. In
	158	terms of :app:`Pyramid`, the current registry is a thread local variable.
	159	Using an API that consults a thread local makes understanding how it works
	160	non-local.
fbfea7	161
96d456	162	You've now bought in to the fact that there's a registry that is just hanging
CM	163	around. But how does the registry get populated? Why, via code that calls
	164	directives like ``config.add_view``. In this particular case, however, the
	165	registration of ``ISettings`` is made by the framework itself under the hood:
	166	it's not present in any user configuration. This is extremely hard to
	167	comprehend. Problem number six.
fbfea7	168
e8609b	169	Clearly there's some amount of cognitive load here that needs to be borne by a
SP	170	reader of code that extends the :app:`Pyramid` framework due to its use of the
	171	ZCA, even if they are already an expert Python programmer and an expert in the
	172	domain of web applications. This is suboptimal.
fbfea7	173
CM	174	Ameliorations
	175	+++++++++++++
	176
8cdb1b	177	First, the primary amelioration: :app:`Pyramid` *does not expect application
2b8094	178	developers to understand ZCA concepts or any of its APIs. If an application*
SP	179	developer needs to understand a ZCA concept or API during the creation of a
	180	:app:`Pyramid` application, we've failed on some axis.
fbfea7	181
2b8094	182	Instead the framework hides the presence of the ZCA registry behind
8cdb1b	183	special-purpose API functions that do use ZCA APIs. Take for example the
CM	184	``pyramid.security.authenticated_userid`` function, which returns the userid
	185	present in the current request or ``None`` if no userid is present in the
	186	current request. The application developer calls it like so:
fbfea7	187
CM	188	.. code-block:: python
4ef406	189	:linenos:
fbfea7	190
4ef406	191	from pyramid.security import authenticated_userid
SP	192	userid = authenticated_userid(request)
fbfea7	193
2b8094	194	They now have the current user id.
fbfea7	195
2b8094	196	Under its hood however, the implementation of ``authenticated_userid`` is this:
fbfea7	197
b7fdea	198	.. code-block:: python
4ef406	199	:linenos:
fbfea7	200
4ef406	201	def authenticated_userid(request):
SP	202	""" Return the userid of the currently authenticated user or
	203	``None`` if there is no authentication policy in effect or there
	204	is no currently authenticated user. """
fbfea7	205
4ef406	206	registry = request.registry # the ZCA component registry
SP	207	policy = registry.queryUtility(IAuthenticationPolicy)
	208	if policy is None:
	209	return None
	210	return policy.authenticated_userid(request)
fbfea7	211
8cdb1b	212	Using such wrappers, we strive to always hide the ZCA API from application
2b8094	213	developers. Application developers should just never know about the ZCA API;
SP	214	they should call a Python function with some object germane to the domain as an
	215	argument, and it should return a result. A corollary that follows is that any
	216	reader of an application that has been written using :app:`Pyramid` needn't
	217	understand the ZCA API either.
fbfea7	218
8cdb1b	219	Hiding the ZCA API from application developers and code readers is a form of
96d456	220	enhancing domain specificity. No application developer wants to need to
2b8094	221	understand the small, detailed mechanics of how a web framework does its thing.
SP	222	People want to deal in concepts that are closer to the domain they're working
	223	in. For example, web developers want to know about users, not utilities.
	224	:app:`Pyramid` uses the ZCA as an implementation detail, not as a feature which
	225	is exposed to end users.
fbfea7	226
8cdb1b	227	However, unlike application developers, framework developers, including
CM	228	people who want to override :app:`Pyramid` functionality via preordained
2b8094	229	framework plugpoints like traversal or view lookup, must understand the ZCA
8cdb1b	230	registry API.
fbfea7	231
8cdb1b	232	:app:`Pyramid` framework developers were so concerned about conceptual load
2b8094	233	issues of the ZCA registry API that a `replacement registry implementation
SP	234	<https://github.com/repoze/repoze.component>`_ named :mod:`repoze.component`
	235	was actually developed. Though this package has a registry implementation
	236	which is fully functional and well-tested, and its API is much nicer than the
	237	ZCA registry API, work on it was largely abandoned, and it is not used in
	238	:app:`Pyramid`. We continued to use a ZCA registry within :app:`Pyramid`
	239	because it ultimately proved a better fit.
aade46	240
012b97	241	.. note::
M	242
2b8094	243	We continued using ZCA registry rather than disusing it in favor of using
SP	244	the registry implementation in :mod:`repoze.component` largely because the
	245	ZCA concept of interfaces provides for use of an interface hierarchy, which
	246	is useful in a lot of scenarios (such as context type inheritance). Coming
	247	up with a marker type that was something like an interface that allowed for
	248	this functionality seemed like it was just reinventing the wheel.
fbfea7	249
2b8094	250	Making framework developers and extenders understand the ZCA registry API is a
SP	251	trade-off. We (the :app:`Pyramid` developers) like the features that the ZCA
	252	registry gives us, and we have long-ago borne the weight of understanding what
	253	it does and how it works. The authors of :app:`Pyramid` understand the ZCA
	254	deeply and can read code that uses it as easily as any other code.
fbfea7	255
ea32d6	256	But we recognize that developers who might want to extend the framework are not
2b8094	257	as comfortable with the ZCA registry API as the original developers. So for
SP	258	the purpose of being kind to third-party :app:`Pyramid` framework developers,
	259	we've drawn some lines in the sand.
fbfea7	260
2b8094	261	In all core code, we've made use of ZCA global API functions, such as
SP	262	``zope.component.getUtility`` and ``zope.component.getAdapter``, the exception
8cdb1b	263	instead of the rule. So instead of:
fbfea7	264
8cdb1b	265	.. code-block:: python
4ef406	266	:linenos:
fbfea7	267
4ef406	268	from pyramid.interfaces import IAuthenticationPolicy
SP	269	from zope.component import getUtility
	270	policy = getUtility(IAuthenticationPolicy)
fbfea7	271
8cdb1b	272	:app:`Pyramid` code will usually do:
fbfea7	273
8cdb1b	274	.. code-block:: python
4ef406	275	:linenos:
fbfea7	276
4ef406	277	from pyramid.interfaces import IAuthenticationPolicy
SP	278	from pyramid.threadlocal import get_current_registry
	279	registry = get_current_registry()
	280	policy = registry.getUtility(IAuthenticationPolicy)
fbfea7	281
2b8094	282	While the latter is more verbose, it also arguably makes it more obvious what's
SP	283	going on. All of the :app:`Pyramid` core code uses this pattern rather than
	284	the ZCA global API.
fbfea7	285
CM	286	Rationale
	287	+++++++++
	288
8cdb1b	289	Here are the main rationales involved in the :app:`Pyramid` decision to use
CM	290	the ZCA registry:
fbfea7	291
d141dd	292	- History. A nontrivial part of the answer to this question is "history".
8cdb1b	293	Much of the design of :app:`Pyramid` is stolen directly from :term:`Zope`.
CM	294	Zope uses the ZCA registry to do a number of tricks. :app:`Pyramid` mimics
	295	these tricks, and, because the ZCA registry works well for that set of
	296	tricks, :app:`Pyramid` uses it for the same purposes. For example, the way
	297	that :app:`Pyramid` maps a :term:`request` to a :term:`view callable` using
	298	:term:`traversal` is lifted almost entirely from Zope. The ZCA registry
	299	plays an important role in the particulars of how this request to view
	300	mapping is done.
fbfea7	301
8cdb1b	302	- Features. The ZCA component registry essentially provides what can be
96d456	303	considered something like a superdictionary, which allows for more complex
CM	304	lookups than retrieving a value based on a single key. Some of this lookup
	305	capability is very useful for end users, such as being able to register a
	306	view that is only found when the context is some class of object, or when
	307	the context implements some :term:`interface`.
fbfea7	308
1d8374	309	- Singularity. There's only one place where "application configuration" lives
SP	310	in a :app:`Pyramid` application: in a component registry. The component
	311	registry answers questions made to it by the framework at runtime based on
	312	the configuration of an application. Note: "an application" is not the
	313	same as "a process"; multiple independently configured copies of the same
	314	:app:`Pyramid` application are capable of running in the same process space.
fbfea7	315
8cdb1b	316	- Composability. A ZCA component registry can be populated imperatively, or
CM	317	there's an existing mechanism to populate a registry via the use of a
55ce9d	318	configuration file (ZCML, via the optional :term:`pyramid_zcml` package).
CM	319	We didn't need to write a frontend from scratch to make use of
	320	configuration-file-driven registry population.
fbfea7	321
8cdb1b	322	- Pluggability. Use of the ZCA registry allows for framework extensibility
CM	323	via a well-defined and widely understood plugin architecture. As long as
	324	framework developers and extenders understand the ZCA registry, it's
	325	possible to extend :app:`Pyramid` almost arbitrarily. For example, it's
96d456	326	relatively easy to build a directive that registers several views all at
CM	327	once, allowing app developers to use that directive as a "macro" in code
bee624	328	that they write. This is somewhat of a differentiating feature from other
CM	329	(non-Zope) frameworks.
fbfea7	330
8cdb1b	331	- Testability. Judicious use of the ZCA registry in framework code makes
1d8374	332	testing that code slightly easier. Instead of using monkeypatching or other
SP	333	facilities to register mock objects for testing, we inject dependencies via
	334	ZCA registrations, then use lookups in the code to find our mock objects.
fbfea7	335
8cdb1b	336	- Speed. The ZCA registry is very fast for a specific set of complex lookup
CM	337	scenarios that :app:`Pyramid` uses, having been optimized through the years
	338	for just these purposes. The ZCA registry contains optional C code for
	339	this purpose which demonstrably has no (or very few) bugs.
fbfea7	340
bee624	341	- Ecosystem. Many existing Zope packages can be used in :app:`Pyramid` with
CM	342	few (or no) changes due to our use of the ZCA registry.
fbfea7	343
CM	344	Conclusion
	345	++++++++++
	346
fd5ae9	347	If you only develop applications using :app:`Pyramid`, there's not much to
1d8374	348	complain about here. You just should never need to understand the ZCA registry
SP	349	API; use documented :app:`Pyramid` APIs instead. However, you may be an
	350	application developer who doesn't read API documentation. Instead you
	351	read the raw source code, and because you haven't read the API documentation,
	352	you don't know what functions, classes, and methods even form the
	353	:app:`Pyramid` API. As a result, you've now written code that uses internals,
	354	and you've painted yourself into a conceptual corner, needing to wrestle with
	355	some ZCA-using implementation detail. If this is you, it's extremely hard to
	356	have a lot of sympathy for you. You'll either need to get familiar with how
	357	we're using the ZCA registry or you'll need to use only the documented APIs;
	358	that's why we document them as APIs.
fbfea7	359
bee624	360	If you extend or develop :app:`Pyramid` (create new directives, use some
96d456	361	of the more obscure hooks as described in :ref:`hooks_chapter`, or work on
bee624	362	the :app:`Pyramid` core code), you will be faced with needing to understand
CM	363	at least some ZCA concepts. In some places it's used unabashedly, and will
	364	be forever. We know it's quirky, but it's also useful and fundamentally
	365	understandable if you take the time to do some reading about it.
1d8374	366
fbfea7	367
b1d4c0	368	.. _zcml_encouragement:
CM	369
edd915	370	Pyramid "Encourages Use of ZCML"
CM	371	--------------------------------
fbfea7	372
809744	373	:term:`ZCML` is a configuration language that can be used to configure the
8cdb1b	374	:term:`Zope Component Architecture` registry that :app:`Pyramid` uses for
809744	375	application configuration. Often people claim that Pyramid "needs ZCML".
fbfea7	376
c9c3c4	377	It doesn't. In :app:`Pyramid` 1.0, ZCML doesn't ship as part of the core;
CM	378	instead it ships in the :term:`pyramid_zcml` add-on package, which is
	379	completely optional. No ZCML is required at all to use :app:`Pyramid`, nor
	380	any other sort of frameworky declarative frontend to application
	381	configuration.
fbfea7	382
1d8374	383
SP	384	Pyramid Does Traversal, and I Don't Like Traversal
edd915	385	--------------------------------------------------
7bb032	386
a5ffd6	387	In :app:`Pyramid`, :term:`traversal` is the act of resolving a URL path to a
1d8374	388	:term:`resource` object in a resource tree. Some people are uncomfortable with
SP	389	this notion, and believe it is wrong. Thankfully if you use :app:`Pyramid` and
	390	you don't want to model your application in terms of a resource tree, you
	391	needn't use it at all. Instead use :term:`URL dispatch` to map URL paths to
	392	views.
c58cf2	393
96d456	394	The idea that some folks believe traversal is unilaterally wrong is
ae1d22	395	understandable. The people who believe it is wrong almost invariably have
CM	396	all of their data in a relational database. Relational databases aren't
96d456	397	naturally hierarchical, so traversing one like a tree is not possible.
c58cf2	398
ae1d22	399	However, folks who deem traversal unilaterally wrong are neglecting to take
CM	400	into account that many persistence mechanisms are hierarchical. Examples
a5ffd6	401	include a filesystem, an LDAP database, a :term:`ZODB` (or another type of
CM	402	graph) database, an XML document, and the Python module namespace. It is
	403	often convenient to model the frontend to a hierarchical data store as a
	404	graph, using traversal to apply views to objects that either are the
	405	resources in the tree being traversed (such as in the case of ZODB) or at
	406	least ones which stand in for them (such as in the case of wrappers for files
	407	from the filesystem).
c58cf2	408
a5ffd6	409	Also, many website structures are naturally hierarchical, even if the data
CM	410	which drives them isn't. For example, newspaper websites are often extremely
	411	hierarchical: sections within sections within sections, ad infinitum. If you
	412	want your URLs to indicate this structure, and the structure is indefinite
	413	(the number of nested sections can be "N" instead of some fixed number), a
	414	resource tree is an excellent way to model this, even if the backend is a
d11d30	415	relational database. In this situation, the resource tree is just a site
a5ffd6	416	structure.
c58cf2	417
ae1d22	418	Traversal also offers better composability of applications than URL dispatch,
CM	419	because it doesn't rely on a fixed ordering of URL matching. You can compose
	420	a set of disparate functionality (and add to it later) around a mapping of
96d456	421	view to resource more predictably than trying to get the right ordering of
ae1d22	422	URL pattern matching.
CM	423
	424	But the point is ultimately moot. If you don't want to use traversal, you
	425	needn't. Use URL dispatch instead.
c58cf2	426
1d8374	427
SP	428	Pyramid Does URL Dispatch, and I Don't Like URL Dispatch
edd915	429	--------------------------------------------------------
c58cf2	430
ae1d22	431	In :app:`Pyramid`, :term:`url dispatch` is the act of resolving a URL path to
CM	432	a :term:`view` callable by performing pattern matching against some set of
	433	ordered route definitions. The route definitions are examined in order: the
	434	first pattern which matches is used to associate the URL with a view
	435	callable.
c58cf2	436
ae1d22	437	Some people are uncomfortable with this notion, and believe it is wrong.
CM	438	These are usually people who are steeped deeply in :term:`Zope`. Zope does
	439	not provide any mechanism except :term:`traversal` to map code to URLs. This
	440	is mainly because Zope effectively requires use of :term:`ZODB`, which is a
	441	hierarchical object store. Zope also supports relational databases, but
	442	typically the code that calls into the database lives somewhere in the ZODB
	443	object graph (or at least is a :term:`view` related to a node in the object
	444	graph), and traversal is required to reach this code.
c58cf2	445
589ee7	446	I'll argue that URL dispatch is ultimately useful, even if you want to use
CM	447	traversal as well. You can actually combine URL dispatch and traversal in
fd5ae9	448	:app:`Pyramid` (see :ref:`hybrid_chapter`). One example of such a usage: if
589ee7	449	you want to emulate something like Zope 2's "Zope Management Interface" UI on
1d8374	450	top of your object graph (or any administrative interface), you can register a
SP	451	route like ``config.add_route('manage', '/manage/*traverse')`` and then
	452	associate "management" views in your code by using the ``route_name`` argument
	453	to a ``view`` configuration, e.g., ``config.add_view('.some.callable',
	454	context=".some.Resource", route_name='manage')``. If you wire things up this
	455	way, someone then walks up to, for example, ``/manage/ob1/ob2``, they might be
	456	presented with a management interface, but walking up to ``/ob1/ob2`` would
	457	present them with the default object view. There are other tricks you can pull
	458	in these hybrid configurations if you're clever (and maybe masochistic) too.
c58cf2	459
1d8374	460	Also, if you are a URL dispatch hater, if you should ever be asked to write an
SP	461	application that must use some legacy relational database structure, you might
	462	find that using URL dispatch comes in handy for one-off associations between
	463	views and URL paths. Sometimes it's just pointless to add a node to the object
	464	graph that effectively represents the entry point for some bit of code. You
	465	can just use a route and be done with it. If a route matches, a view
	466	associated with the route will be called. If no route matches, :app:`Pyramid`
	467	falls back to using traversal.
c58cf2	468
ae1d22	469	But the point is ultimately moot. If you use :app:`Pyramid`, and you really
CM	470	don't want to use URL dispatch, you needn't use it at all. Instead, use
	471	:term:`traversal` exclusively to map URL paths to views, just like you do in
	472	:term:`Zope`.
c58cf2	473
1d8374	474
edd915	475	Pyramid Views Do Not Accept Arbitrary Keyword Arguments
CM	476	-------------------------------------------------------
f34859	477
589ee7	478	Many web frameworks (Zope, TurboGears, Pylons 1.X, Django) allow for their
CM	479	variant of a :term:`view callable` to accept arbitrary keyword or positional
51573a	480	arguments, which are filled in using values present in the ``request.POST``,
SP	481	``request.GET``, or route match dictionaries. For example, a Django view will
	482	accept positional arguments which match information in an associated "urlconf"
	483	such as ``r'^polls/(?P<poll_id>\d+)/$``:
f34859	484
CM	485	.. code-block:: python
4ef406	486	:linenos:
f34859	487
4ef406	488	def aview(request, poll_id):
SP	489	return HttpResponse(poll_id)
f34859	490
51573a	491	Zope likewise allows you to add arbitrary keyword and positional arguments to
SP	492	any method of a resource object found via traversal:
f34859	493
CM	494	.. code-block:: python
4ef406	495	:linenos:
0ac7b0	496
4ef406	497	from persistent import Persistent
f34859	498
4ef406	499	class MyZopeObject(Persistent):
f34859	500	def aview(self, a, b, c=None):
CM	501	return '%s %s %c' % (a, b, c)
	502
ae1d22	503	When this method is called as the result of being the published callable, the
CM	504	Zope request object's GET and POST namespaces are searched for keys which
	505	match the names of the positional and keyword arguments in the request, and
	506	the method is called (if possible) with its argument list filled with values
	507	mentioned therein. TurboGears and Pylons 1.X operate similarly.
f34859	508
51573a	509	Out of the box, :app:`Pyramid` is configured to have none of these features. By
SP	510	default :app:`Pyramid` view callables always accept only ``request`` and no
	511	other arguments. The rationale is, this argument specification matching when
	512	done aggressively can be costly, and :app:`Pyramid` has performance as one of
	513	its main goals. Therefore we've decided to make people, by default, obtain
	514	information by interrogating the request object within the view callable body
	515	instead of providing magic to do unpacking into the view argument list.
f34859	516
ae1d22	517	However, as of :app:`Pyramid` 1.0a9, user code can influence the way view
CM	518	callables are expected to be called, making it possible to compose a system
	519	out of view callables which are called with arbitrary arguments. See
	520	:ref:`using_a_view_mapper`.
f34859	521
edd915	522	Pyramid Provides Too Few "Rails"
CM	523	--------------------------------
6956a8	524
96d456	525	By design, :app:`Pyramid` is not a particularly opinionated web framework.
f52ff2	526	It has a relatively parsimonious feature set. It contains no built in ORM
CM	527	nor any particular database bindings. It contains no form generation
	528	framework. It has no administrative web user interface. It has no built in
	529	text indexing. It does not dictate how you arrange your code.
6956a8	530
f52ff2	531	Such opinionated functionality exists in applications and frameworks built
fd5ae9	532	on top of :app:`Pyramid`. It's intended that higher-level systems emerge
2033ee	533	built using :app:`Pyramid` as a base.
SP	534
	535	.. seealso::
	536
	537	See also :ref:`apps_are_extensible`.
6956a8	538
edd915	539	Pyramid Provides Too Many "Rails"
CM	540	---------------------------------
6956a8	541
a09149	542	:app:`Pyramid` provides some features that other web frameworks do not.
CM	543	These are features meant for use cases that might not make sense to you if
96d456	544	you're building a simple bespoke web application:
6956a8	545
a09149	546	- An optional way to map URLs to code using :term:`traversal` which implies a
CM	547	walk of a :term:`resource tree`.
6956a8	548
a09149	549	- The ability to aggregate Pyramid application configuration from multiple
CM	550	sources using :meth:`pyramid.config.Configurator.include`.
6956a8	551
a09149	552	- View and subscriber registrations made using :term:`interface` objects
b53bff	553	instead of class objects (e.g., :ref:`using_resource_interfaces`).
a09149	554
CM	555	- A declarative :term:`authorization` system.
	556
	557	- Multiple separate I18N :term:`translation string` factories, each of which
96d456	558	can name its own domain.
a09149	559
CM	560	These features are important to the authors of :app:`Pyramid`. The
	561	:app:`Pyramid` authors are often commissioned to build CMS-style
96d456	562	applications. Such applications are often frameworky because they have more
CM	563	than one deployment. Each deployment requires a slightly different
ae1d22	564	composition of sub-applications, and the framework and sub-applications often
a09149	565	need to be extensible. Because the application has more than one
CM	566	deployment, pluggability and extensibility is important, as maintaining
	567	multiple forks of the application, one per deployment, is extremely
	568	undesirable. Because it's easier to extend a system that uses
96d456	569	:term:`traversal` from the outside than it is to do the same in a system that
CM	570	uses :term:`URL dispatch`, each deployment uses a :term:`resource tree`
a09149	571	composed of a persistent tree of domain model objects, and uses
CM	572	:term:`traversal` to map :term:`view callable` code to resources in the tree.
	573	The resource tree contains very granular security declarations, as resources
ae1d22	574	are owned and accessible by different sets of users. Interfaces are used to
CM	575	make unit testing and implementation substitutability easier.
a09149	576
CM	577	In a bespoke web application, usually there's a single canonical deployment,
	578	and therefore no possibility of multiple code forks. Extensibility is not
b53bff	579	required; the code is just changed in place. Security requirements are often
SP	580	less granular. Using the features listed above will often be overkill for such
	581	an application.
a09149	582
CM	583	If you don't like these features, it doesn't mean you can't or shouldn't use
b53bff	584	:app:`Pyramid`. They are all optional, and a lot of time has been spent making
SP	585	sure you don't need to know about them up front. You can build "Pylons 1.X
	586	style" applications using :app:`Pyramid` that are purely bespoke by ignoring
	587	the features above. You may find these features handy later after building a
	588	bespoke web application that suddenly becomes popular and requires
	589	extensibility because it must be deployed in multiple locations.
6956a8	590
edd915	591	Pyramid Is Too Big
CM	592	------------------
4b32c8	593
af33f7	594	"The :app:`Pyramid` compressed tarball is larger than 2MB. It must be enormous!"
4b32c8	595
b53bff	596	No. We just ship it with docs, test code, and scaffolding. Here's a breakdown
SP	597	of what's included in subdirectories of the package tree:
4b32c8	598
CM	599	docs/
	600
b53bff	601	3.6MB
4b32c8	602
5c1510	603	pyramid/tests/
4b32c8	604
b53bff	605	1.3MB
4b32c8	606
5614e9	607	pyramid/scaffolds/
4b32c8	608
b53bff	609	133KB
4b32c8	610
1ce7fe	611	pyramid/ (except for ``pyramid/tests`` and ``pyramid/scaffolds``)
4b32c8	612
b53bff	613	812KB
4b32c8	614
5614e9	615	Of the approximately 34K lines of Python code in the package, the code
589ee7	616	that actually has a chance of executing during normal operation, excluding
643b2a	617	tests and scaffolding Python files, accounts for approximately 10K lines.
CM	618
4b32c8	619
edd915	620	Pyramid Has Too Many Dependencies
CM	621	---------------------------------
4b32c8	622
dd5358	623	Over time, we've made lots of progress on reducing the number of packaging
CM	624	dependencies Pyramid has had. Pyramid 1.2 had 15 of them. Pyramid 1.3 and 1.4
	625	had 12 of them. The current release as of this writing, Pyramid 1.5, has
	626	only 7. This number is unlikely to become any smaller.
4b32c8	627
dd5358	628	A port to Python 3 completed in Pyramid 1.3 helped us shed a good number of
CM	629	dependencies by forcing us to make better packaging decisions. Removing
	630	Chameleon and Mako templating system dependencies in the Pyramid core in 1.5
	631	let us shed most of the remainder of them.
93f694	632
41aa60	633
SP	634	Pyramid "Cheats" to Obtain Speed
edd915	635	--------------------------------
93f694	636
8cdb1b	637	Complaints have been lodged by other web framework authors at various times
CM	638	that :app:`Pyramid` "cheats" to gain performance. One claimed cheating
	639	mechanism is our use (transitively) of the C extensions provided by
	640	:mod:`zope.interface` to do fast lookups. Another claimed cheating mechanism
	641	is the religious avoidance of extraneous function calls.
93f694	642
41aa60	643	If there's such a thing as cheating to get better performance, we want to cheat
SP	644	as much as possible. We optimize :app:`Pyramid` aggressively. This comes at a
	645	cost. The core code has sections that could be expressed with more readability.
	646	As an amelioration, we've commented these sections liberally.
	647
4b32c8	648
edd915	649	Pyramid Gets Its Terminology Wrong ("MVC")
CM	650	------------------------------------------
384ead	651
8cdb1b	652	"I'm a MVC web framework user, and I'm confused. :app:`Pyramid` calls the
CM	653	controller a view! And it doesn't have any controllers."
384ead	654
5df6be	655	If you are in this camp, you might have come to expect things about how your
CM	656	existing "MVC" framework uses its terminology. For example, you probably
	657	expect that models are ORM models, controllers are classes that have methods
fd5ae9	658	that map to URLs, and views are templates. :app:`Pyramid` indeed has each of
5df6be	659	these concepts, and each probably works almost exactly like your existing
41aa60	660	"MVC" web framework. We just don't use the MVC terminology, as we can't square
SP	661	its usage in the web framework space with historical reality.
5df6be	662
8cdb1b	663	People very much want to give web applications the same properties as common
CM	664	desktop GUI platforms by using similar terminology, and to provide some frame
	665	of reference for how various components in the common web framework might
	666	hang together. But in the opinion of the author, "MVC" doesn't match the web
	667	very well in general. Quoting from the `Model-View-Controller Wikipedia entry
1cb30e	668	<https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller>`_:
16cd50	669
41aa60	670	Though MVC comes in different flavors, control flow is generally as
SP	671	follows:
384ead	672
41aa60	673	The user interacts with the user interface in some way (for example,
SP	674	presses a mouse button).
384ead	675
41aa60	676	The controller handles the input event from the user interface, often via
SP	677	a registered handler or callback and converts the event into appropriate
	678	user action, understandable for the model.
384ead	679
41aa60	680	The controller notifies the model of the user action, possibly resulting
SP	681	in a change in the model's state. (For example, the controller updates the
	682	user's shopping cart.)[5]
384ead	683
41aa60	684	A view queries the model in order to generate an appropriate user
SP	685	interface (for example, the view lists the shopping cart's contents). Note
	686	that the view gets its own data from the model.
384ead	687
41aa60	688	The controller may (in some implementations) issue a general instruction
SP	689	to the view to render itself. In others, the view is automatically
	690	notified by the model of changes in state (Observer) which require a
	691	screen update.
384ead	692
41aa60	693	The user interface waits for further user interactions, which restarts the
SP	694	cycle.
384ead	695
8cdb1b	696	To the author, it seems as if someone edited this Wikipedia definition,
CM	697	tortuously couching concepts in the most generic terms possible in order to
41aa60	698	account for the use of the term "MVC" by current web frameworks. I doubt such
SP	699	a broad definition would ever be agreed to by the original authors of the MVC
	700	pattern. But even so, it seems most MVC web frameworks fail to meet even
	701	this falsely generic definition.
384ead	702
8cdb1b	703	For example, do your templates (views) always query models directly as is
41aa60	704	claimed in "note that the view gets its own data from the model"? Probably not.
SP	705	My "controllers" tend to do this, massaging the data for easier use by the
	706	"view" (template). What do you do when your "controller" returns JSON? Do your
	707	controllers use a template to generate JSON? If not, what's the "view" then?
	708	Most MVC-style GUI web frameworks have some sort of event system hooked up that
	709	lets the view detect when the model changes. The web just has no such facility
	710	in its current form; it's effectively pull-only.
384ead	711
41aa60	712	So, in the interest of not mistaking desire with reality, and instead of trying
SP	713	to jam the square peg that is the web into the round hole of "MVC", we just
	714	punt and say there are two things: resources and views. The resource tree
	715	represents a site structure, the view presents a resource. The templates are
	716	really just an implementation detail of any given view. A view doesn't need a
	717	template to return a response. There's no "controller"; it just doesn't exist.
	718	The "model" is either represented by the resource tree or by a "domain model"
	719	(like an SQLAlchemy model) that is separate from the framework entirely. This
	720	seems to us like more reasonable terminology, given the current constraints of
	721	the web.
	722
384ead	723
0435ce	724	.. _apps_are_extensible:
CM	725
628dac	726	Pyramid Applications Are Extensible; I Don't Believe in Application Extensibility
edd915	727	---------------------------------------------------------------------------------
1df991	728
8cdb1b	729	Any :app:`Pyramid` application written obeying certain constraints is
CM	730	extensible. This feature is discussed in the :app:`Pyramid` documentation
628dac	731	chapters named :ref:`extending_chapter` and :ref:`advconfig_narr`. It is made
SP	732	possible by the use of the :term:`Zope Component Architecture` within
	733	:app:`Pyramid`.
ddb6a8	734
628dac	735	"Extensible" in this context means:
1df991	736
628dac	737	- The behavior of an application can be overridden or extended in a particular
SP	738	deployment of the application without requiring that the deployer modify
	739	the source of the original application.
1df991	740
628dac	741	- The original developer is not required to anticipate any extensibility
SP	742	plug points at application creation time to allow fundamental application
	743	behavior to be overridden or extended.
1df991	744
ddb6a8	745	- The original developer may optionally choose to anticipate an
628dac	746	application-specific set of plug points, which may be hooked by a deployer.
SP	747	If they choose to use the facilities provided by the ZCA, the original
	748	developer does not need to think terribly hard about the mechanics of
	749	introducing such a plug point.
1df991	750
96d456	751	Many developers seem to believe that creating extensible applications is not
628dac	752	worth it. They instead suggest that modifying the source of a given application
SP	753	for each deployment to override behavior is more reasonable. Much discussion
	754	about version control branching and merging typically ensues.
1df991	755
628dac	756	It's clear that making every application extensible isn't required. The
SP	757	majority of web applications only have a single deployment, and thus needn't be
	758	extensible at all. However some web applications have multiple deployments, and
	759	others have many deployments. For example, a generic content management
	760	system (CMS) may have basic functionality that needs to be extended for a
	761	particular deployment. That CMS may be deployed for many organizations at many
	762	places. Some number of deployments of this CMS may be deployed centrally by a
	763	third party and managed as a group. It's easier to be able to extend such a
	764	system for each deployment via preordained plug points than it is to
	765	continually keep each software branch of the system in sync with some upstream
	766	source. The upstream developers may change code in such a way that your changes
	767	to the same codebase conflict with theirs in fiddly, trivial ways. Merging such
	768	changes repeatedly over the lifetime of a deployment can be difficult and time
	769	consuming, and it's often useful to be able to modify an application for a
	770	particular deployment in a less invasive way.
1df991	771
8cdb1b	772	If you don't want to think about :app:`Pyramid` application extensibility at
628dac	773	all, you needn't. You can ignore extensibility entirely. However if you follow
SP	774	the set of rules defined in :ref:`extending_chapter`, you don't need to make
	775	your application extensible. Any application you write in the framework just
	776	is automatically extensible at a basic level. The mechanisms that deployers
	777	use to extend it will be necessarily coarse. Typically views, routes, and
	778	resources will be capable of being overridden. But for most minor (and even
	779	some major) customizations, these are often the only override plug points
	780	necessary. If the application doesn't do exactly what the deployment requires,
	781	it's often possible for a deployer to override a view, route, or resource, and
	782	quickly make it do what they want it to do in ways *not necessarily anticipated
	783	by the original developer*. Here are some example scenarios demonstrating the
	784	benefits of such a feature.
1df991	785
628dac	786	- If a deployment needs a different styling, the deployer may override the main
SP	787	template and the CSS in a separate Python package which defines overrides.
f5d708	788
628dac	789	- If a deployment needs an application page to do something differently, or to
SP	790	expose more or different information, the deployer may override the view that
	791	renders the page within a separate Python package.
f5d708	792
8cdb1b	793	- If a deployment needs an additional feature, the deployer may add a view to
CM	794	the override package.
f5d708	795
628dac	796	As long as the fundamental design of the upstream package doesn't change, these
SP	797	types of modifications often survive across many releases of the upstream
	798	package without needing to be revisited.
ddb6a8	799
8cdb1b	800	Extending an application externally is not a panacea, and carries a set of
628dac	801	risks similar to branching and merging. Sometimes major changes upstream will
SP	802	cause you to revisit and update some of your modifications. But you won't
	803	regularly need to deal with meaningless textual merge conflicts that trivial
	804	changes to upstream packages often entail when it comes time to update the
	805	upstream package, because if you extend an application externally, there just
	806	is no textual merge done. Your modifications will also, for whatever it's
	807	worth, be contained in one, canonical, well-defined place.
ddb6a8	808
8cdb1b	809	Branching an application and continually merging in order to get new features
628dac	810	and bug fixes is clearly useful. You can do that with a :app:`Pyramid`
SP	811	application just as usefully as you can do it with any application. But
8cdb1b	812	deployment of an application written in :app:`Pyramid` makes it possible to
628dac	813	avoid the need for this even if the application doesn't define any plug points
SP	814	ahead of time. It's possible that promoters of competing web frameworks dismiss
	815	this feature in favor of branching and merging because applications written in
	816	their framework of choice aren't extensible out of the box in a comparably
	817	fundamental way.
f5d708	818
2d2b43	819	While :app:`Pyramid` applications are fundamentally extensible even if you
8cdb1b	820	don't write them with specific extensibility in mind, if you're moderately
628dac	821	adventurous, you can also take it a step further. If you learn more about the
SP	822	:term:`Zope Component Architecture`, you can optionally use it to expose other
	823	more domain-specific configuration plug points while developing an application.
	824	The plug points you expose needn't be as coarse as the ones provided
	825	automatically by :app:`Pyramid` itself. For example, you might compose your own
	826	directive that configures a set of views for a pre-baked purpose (e.g.,
	827	``restview`` or somesuch), allowing other people to refer to that directive
	828	when they make declarations in the ``includeme`` of their customization
	829	package. There is a cost for this: the developer of an application that defines
	830	custom plug points for its deployers will need to understand the ZCA or they
	831	will need to develop their own similar extensibility system.
1df991	832
628dac	833	Ultimately any argument about whether the extensibility features lent to
SP	834	applications by :app:`Pyramid` are good or bad is mostly pointless. You needn't
	835	take advantage of the extensibility features provided by a particular
8cdb1b	836	:app:`Pyramid` application in order to affect a modification for a particular
628dac	837	set of its deployments. You can ignore the application's extensibility plug
SP	838	points entirely, and use version control branching and merging to manage
	839	application deployment modifications instead, as if you were deploying an
	840	application written using any other web framework.
	841
15be26	842
4df9a0	843	Zope 3 Enforces "TTW" Authorization Checks by Default; Pyramid Does Not
edd915	844	-----------------------------------------------------------------------
a39aca	845
CM	846	Challenge
672bbe	847	+++++++++
a39aca	848
8cdb1b	849	:app:`Pyramid` performs automatic authorization checks only at :term:`view`
1cb30e	850	execution time. Zope 3 wraps context objects with a security proxy, which
SP	851	causes Zope 3 also to do security checks during attribute access. I like this,
	852	because it means:
a39aca	853
CM	854	#) When I use the security proxy machinery, I can have a view that
	855	conditionally displays certain HTML elements (like form fields) or
08c221	856	prevents certain attributes from being modified depending on the
0d43ed	857	permissions that the accessing user possesses with respect to a context
CM	858	object.
a39aca	859
a5ffd6	860	#) I want to also expose my resources via a REST API using Twisted Web. If
0d43ed	861	Pyramid performed authorization based on attribute access via Zope3's
a5ffd6	862	security proxies, I could enforce my authorization policy in both
fd5ae9	863	:app:`Pyramid` and in the Twisted-based system the same way.
a39aca	864
CM	865	Defense
672bbe	866	+++++++
a39aca	867
8cdb1b	868	:app:`Pyramid` was developed by folks familiar with Zope 2, which has a
96d456	869	"through the web" security model. This TTW security model was the precursor
CM	870	to Zope 3's security proxies. Over time, as the :app:`Pyramid` developers
	871	(working in Zope 2) created such sites, we found authorization checks during
	872	code interpretation extremely useful in a minority of projects. But much of
	873	the time, TTW authorization checks usually slowed down the development
	874	velocity of projects that had no delegation requirements. In particular, if
	875	we weren't allowing untrusted users to write arbitrary Python code to be
	876	executed by our application, the burden of through the web security checks
	877	proved too costly to justify. We (collectively) haven't written an
	878	application on top of which untrusted developers are allowed to write code in
	879	many years, so it seemed to make sense to drop this model by default in a new
	880	web framework.
a39aca	881
8cdb1b	882	And since we tend to use the same toolkit for all web applications, it's just
CM	883	never been a concern to be able to use the same set of restricted-execution
4df9a0	884	code under two different web frameworks.
a39aca	885
8cdb1b	886	Justifications for disabling security proxies by default notwithstanding,
96d456	887	given that Zope 3 security proxies are viral by nature, the only requirement
CM	888	to use one is to make sure you wrap a single object in a security proxy and
	889	make sure to access that object normally when you want proxy security checks
	890	to happen. It is possible to override the :app:`Pyramid` traverser for a
	891	given application (see :ref:`changing_the_traverser`). To get Zope3-like
	892	behavior, it is possible to plug in a different traverser which returns
	893	Zope3-security-proxy-wrapped objects for each traversed object (including the
	894	:term:`context` and the :term:`root`). This would have the effect of
	895	creating a more Zope3-like environment without much effort.
a39aca	896
4df9a0	897
8cbbd9	898	.. _http_exception_hierarchy:
CM	899
a3f0a3	900	Pyramid uses its own HTTP exception class hierarchy rather than :mod:`webob.exc`
SP	901	--------------------------------------------------------------------------------
1e5e31	902
40dbf4	903	.. versionadded:: 1.1
1e5e31	904
CM	905	The HTTP exception classes defined in :mod:`pyramid.httpexceptions` are very
a3f0a3	906	much like the ones defined in :mod:`webob.exc`, (e.g.,
SP	907	:class:`~pyramid.httpexceptions.HTTPNotFound` or
	908	:class:`~pyramid.httpexceptions.HTTPForbidden`). They have the same names and
	909	largely the same behavior, and all have a very similar implementation, but not
e636ee	910	the same identity. Here's why they have a separate identity.
1e5e31	911
CM	912	- Making them separate allows the HTTP exception classes to subclass
	913	:class:`pyramid.response.Response`. This speeds up response generation
e636ee	914	slightly due to the way the Pyramid router works. The same speed up could be
a3f0a3	915	gained by monkeypatching :class:`webob.response.Response`, but it's usually
1e5e31	916	the case that monkeypatching turns out to be evil and wrong.
CM	917
e636ee	918	- Making them separate allows them to provide alternate ``__call__`` logic,
1e5e31	919	which also speeds up response generation.
CM	920
	921	- Making them separate allows the exception classes to provide for the proper
	922	value of ``RequestClass`` (:class:`pyramid.request.Request`).
	923
e636ee	924	- Making them separate gives us freedom from thinking about backwards
SP	925	compatibility code present in :mod:`webob.exc` related to Python 2.4, which
	926	we no longer support in Pyramid 1.1+.
1e5e31	927
CM	928	- We change the behavior of two classes
	929	(:class:`~pyramid.httpexceptions.HTTPNotFound` and
	930	:class:`~pyramid.httpexceptions.HTTPForbidden`) in the module so that they
e636ee	931	can be used by Pyramid internally for ``notfound`` and ``forbidden``
SP	932	exceptions.
1e5e31	933
CM	934	- Making them separate allows us to influence the docstrings of the exception
	935	classes to provide Pyramid-specific documentation.
	936
a3f0a3	937	- Making them separate allows us to silence a stupid deprecation warning under
SP	938	Python 2.6 when the response objects are used as exceptions (related to
	939	``self.message``).
1e5e31	940
e636ee	941
8f521b	942	.. _simpler_traversal_model:
CM	943
7045a9	944	Pyramid has simpler traversal machinery than does Zope
8f521b	945	------------------------------------------------------
CM	946
	947	Zope's default traverser:
	948
	949	- Allows developers to mutate the traversal name stack while traversing (they
	950	can add and remove path elements).
	951
96d456	952	- Attempts to use an adaptation to obtain the next element in the path from
8f521b	953	the currently traversed object, falling back to ``__bobo_traverse__``,
7045a9	954	``__getitem__``, and eventually ``__getattr__``.
8f521b	955
CM	956	Zope's default traverser allows developers to mutate the traversal name stack
7045a9	957	during traversal by mutating ``REQUEST['TraversalNameStack']``. Pyramid's
SP	958	default traverser (``pyramid.traversal.ResourceTreeTraverser``) does not offer
	959	a way to do this. It does not maintain a stack as a request attribute and, even
	960	if it did, it does not pass the request to resource objects while it's
	961	traversing. While it was handy at times, this feature was abused in frameworks
	962	built atop Zope (like CMF and Plone), often making it difficult to tell exactly
	963	what was happening when a traversal didn't match a view. I felt it was better
	964	for folks that wanted the feature to make them replace the traverser rather
	965	than build that particular honey pot in to the default traverser.
8f521b	966
CM	967	Zope uses multiple mechanisms to attempt to obtain the next element in the
	968	resource tree based on a name. It first tries an adaptation of the current
7045a9	969	resource to ``ITraversable``, and if that fails, it falls back to attempting a
8f521b	970	number of magic methods on the resource (``__bobo_traverse__``,
7045a9	971	``__getitem__``, and ``__getattr__``). My experience while both using Zope and
SP	972	attempting to reimplement its publisher in ``repoze.zope2`` led me to believe
	973	the following:
8f521b	974
CM	975	- The default traverser should be as simple as possible. Zope's publisher
	976	is somewhat difficult to follow and replicate due to the fallbacks it tried
	977	when one traversal method failed. It is also slow.
	978
	979	- The entire traverser should be replaceable, not just elements of the
96d456	980	traversal machinery. Pyramid has a few big components rather than a
8f521b	981	plethora of small ones. If the entire traverser is replaceable, it's an
CM	982	antipattern to make portions of the default traverser replaceable. Doing
	983	so is a "knobs on knobs" pattern, which is unfortunately somewhat endemic
	984	in Zope. In a "knobs on knobs" pattern, a replaceable subcomponent of a
	985	larger component is made configurable using the same configuration
	986	mechanism that can be used to replace the larger component. For example,
	987	in Zope, you can replace the default traverser by registering an adapter.
	988	But you can also (or alternately) control how the default traverser
	989	traverses by registering one or more adapters. As a result of being able
	990	to either replace the larger component entirely or turn knobs on the
	991	default implementation of the larger component, no one understands when (or
	992	whether) they should ever override the larger component entrirely. This
96d456	993	results, over time, in a rusting together of the larger "replaceable"
7045a9	994	component and the framework itself because people come to depend on the
8f521b	995	availability of the default component in order just to turn its knobs. The
CM	996	default component effectively becomes part of the framework, which entirely
	997	subverts the goal of making it replaceable. In Pyramid, typically if a
96d456	998	component is replaceable, it will itself have no knobs (it will be solid
CM	999	state). If you want to influence behavior controlled by that component,
8f521b	1000	you will replace the component instead of turning knobs attached to the
CM	1001	component.
	1002
7045a9	1003
7b511a	1004	.. _microframeworks_smaller_hello_world:
CM	1005
ed5209	1006	Microframeworks have smaller Hello World programs
7b511a	1007	-------------------------------------------------
CM	1008
1cb30e	1009	Self-described "microframeworks" exist. `Bottle
SP	1010	<http://bottlepy.org/docs/dev/index.html>`_ and `Flask
	1011	<http://flask.pocoo.org/>`_ are two that are becoming popular. `Bobo
5a43e6	1012	<https://bobo.readthedocs.io/en/latest/>`_ doesn't describe itself as a
1cb30e	1013	microframework, but its intended user base is much the same. Many others exist.
SP	1014	We've even (only as a teaching tool, not as any sort of official project)
	1015	`created one using Pyramid <http://static.repoze.org/casts/videotags.html>`_.
	1016	The videos use BFG, a precursor to Pyramid, but the resulting code is
	1017	`available for Pyramid too <https://github.com/Pylons/groundhog>`_).
	1018	Microframeworks are small frameworks with one common feature: each allows its
	1019	users to create a fully functional application that lives in a single Python
	1020	file.
7b511a	1021
8cdb1b	1022	Some developers and microframework authors point out that Pyramid's "hello
ed5209	1023	world" single-file program is longer (by about five lines) than the equivalent
SP	1024	program in their favorite microframework. Guilty as charged.
26a0fd	1025
ed5209	1026	This loss isn't for lack of trying. Pyramid is useful in the same circumstance
SP	1027	in which microframeworks claim dominance: single-file applications. But Pyramid
	1028	doesn't sacrifice its ability to credibly support larger applications in order
	1029	to achieve "hello world" lines of code parity with the current crop of
	1030	microframeworks. Pyramid's design instead tries to avoid some common pitfalls
	1031	associated with naive declarative configuration schemes. The subsections which
	1032	follow explain the rationale.
	1033
7b511a	1034
CM	1035	.. _you_dont_own_modulescope:
	1036
46f74d	1037	Application programmers don't control the module-scope codepath (import-time side-effects are evil)
7b511a	1038	+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
CM	1039
46f74d	1040	Imagine a directory structure with a set of Python files in it:
7b511a	1041
CM	1042	.. code-block:: text
	1043
	1044	.
	1045	\|-- app.py
	1046	\|-- app2.py
	1047	`-- config.py
	1048
	1049	The contents of ``app.py``:
	1050
	1051	.. code-block:: python
16cd50	1052	:linenos:
7b511a	1053
CM	1054	from config import decorator
	1055	from config import L
	1056	import pprint
	1057
	1058	@decorator
	1059	def foo():
	1060	pass
	1061
	1062	if __name__ == '__main__':
	1063	import app2
	1064	pprint.pprint(L)
	1065
	1066	The contents of ``app2.py``:
	1067
	1068	.. code-block:: python
16cd50	1069	:linenos:
7b511a	1070
CM	1071	import app
	1072
	1073	@app.decorator
	1074	def bar():
	1075	pass
	1076
	1077	The contents of ``config.py``:
	1078
	1079	.. code-block:: python
392a6c	1080	:linenos:
7b511a	1081
CM	1082	L = []
	1083
	1084	def decorator(func):
	1085	L.append(func)
	1086	return func
	1087
46f74d	1088	If we ``cd`` to the directory that holds these files, and we run
SP	1089	``python app.py``, given the directory structure and code above, what happens?
	1090	Presumably, our ``decorator`` decorator will be used twice, once by the
	1091	decorated function ``foo`` in ``app.py``, and once by the decorated function
	1092	``bar`` in ``app2.py``. Since each time the decorator is used, the list ``L``
	1093	in ``config.py`` is appended to, we'd expect a list with two elements to be
	1094	printed, right? Sadly, no:
7b511a	1095
16cd50	1096	.. code-block:: text
7b511a	1097
CM	1098	[chrism@thinko]$ python app.py
	1099	[<function foo at 0x7f4ea41ab1b8>,
	1100	<function foo at 0x7f4ea41ab230>,
	1101	<function bar at 0x7f4ea41ab2a8>]
	1102
8cdb1b	1103	By visual inspection, that outcome (three different functions in the list)
46f74d	1104	seems impossible. We defined only two functions, and we decorated each of those
SP	1105	functions only once, so we believe that the ``decorator`` decorator will run
	1106	only twice. However, what we believe is in fact wrong, because the code at
	1107	module scope in our ``app.py`` module was executed twice. The code is
8cdb1b	1108	executed once when the script is run as ``__main__`` (via ``python app.py``),
CM	1109	and then it is executed again when ``app2.py`` imports the same file as
	1110	``app``.
7b511a	1111
46f74d	1112	What does this have to do with our comparison to microframeworks? Many
SP	1113	microframeworks in the current crop (e.g., Bottle and Flask) encourage you to
	1114	attach configuration decorators to objects defined at module scope. These
	1115	decorators execute arbitrarily complex registration code, which populates a
	1116	singleton registry that is a global which is in turn defined in external Python
	1117	module. This is analogous to the above example: the "global registry" in the
	1118	above example is the list ``L``.
7b511a	1119
8cdb1b	1120	Let's see what happens when we use the same pattern with the `Groundhog
965fe1	1121	<https://github.com/Pylons/groundhog>`_ microframework. Replace the contents
CM	1122	of ``app.py`` above with this:
7b511a	1123
CM	1124	.. code-block:: python
16cd50	1125	:linenos:
7b511a	1126
CM	1127	from config import gh
	1128
	1129	@gh.route('/foo/')
	1130	def foo():
	1131	return 'foo'
	1132
	1133	if __name__ == '__main__':
	1134	import app2
	1135	pprint.pprint(L)
	1136
	1137	Replace the contents of ``app2.py`` above with this:
	1138
	1139	.. code-block:: python
16cd50	1140	:linenos:
7b511a	1141
CM	1142	import app
	1143
	1144	@app.gh.route('/bar/')
	1145	def bar():
	1146	'return bar'
	1147
	1148	Replace the contents of ``config.py`` above with this:
	1149
	1150	.. code-block:: python
16cd50	1151	:linenos:
7b511a	1152
CM	1153	from groundhog import Groundhog
	1154	gh = Groundhog('myapp', 'seekrit')
	1155
8cdb1b	1156	How many routes will be registered within the routing table of the "gh"
CM	1157	Groundhog application? If you answered three, you are correct. How many
	1158	would a casual reader (and any sane developer) expect to be registered? If
	1159	you answered two, you are correct. Will the double registration be a
965fe1	1160	problem? With our Groundhog framework's ``route`` method backing this
CM	1161	application, not really. It will slow the application down a little bit,
	1162	because it will need to miss twice for a route when it does not match. Will
	1163	it be a problem with another framework, another application, or another
8cdb1b	1164	decorator? Who knows. You need to understand the application in its
CM	1165	totality, the framework in its totality, and the chronology of execution to
	1166	be able to predict what the impact of unintentional code double-execution
	1167	will be.
7b511a	1168
8cdb1b	1169	The encouragement to use decorators which perform population of an external
CM	1170	registry has an unintended consequence: the application developer now must
46f74d	1171	assert ownership of every code path that executes Python module scope code.
SP	1172	Module-scope code is presumed by the current crop of decorator-based
	1173	microframeworks to execute once and only once. If it executes more than once,
	1174	weird things will start to happen. It is up to the application developer to
	1175	maintain this invariant. Unfortunately, in reality this is an impossible task,
	1176	because Python programmers *do not own the module scope code path, and never
	1177	will*. Anyone who tries to sell you on the idea that they do so is simply
	1178	mistaken. Test runners that you may want to use to run your code's tests often
	1179	perform imports of arbitrary code in strange orders that manifest bugs like the
	1180	one demonstrated above. API documentation generation tools do the same. Some
	1181	people even think it's safe to use the Python ``reload`` command, or delete
	1182	objects from ``sys.modules``, each of which has hilarious effects when used
	1183	against code that has import-time side effects.
7b511a	1184
46f74d	1185	Global registry-mutating microframework programmers therefore will at some
SP	1186	point need to start reading the tea leaves about what might happen if module
	1187	scope code gets executed more than once, like we do in the previous paragraph.
	1188	When Python programmers assume they can use the module-scope code path to run
	1189	arbitrary code (especially code which populates an external registry), and this
	1190	assumption is challenged by reality, the application developer is often
	1191	required to undergo a painful, meticulous debugging process to find the root
	1192	cause of an inevitably obscure symptom. The solution is often to rearrange
	1193	application import ordering, or move an import statement from module-scope into
	1194	a function body. The rationale for doing so can never be expressed adequately
	1195	in the commit message which accompanies the fix, and can't be documented
	1196	succinctly enough for the benefit of the rest of the development team so that
	1197	the problem never happens again. It will happen again, especially if you are
	1198	working on a project with other people who haven't yet internalized the lessons
	1199	you learned while you stepped through module-scope code using ``pdb``. This is
	1200	a very poor situation in which to find yourself as an application developer:
	1201	you probably didn't even know you or your team signed up for the job, because
	1202	the documentation offered by decorator-based microframeworks don't warn you
	1203	about it.
7b511a	1204
8cdb1b	1205	Folks who have a large investment in eager decorator-based configuration that
CM	1206	populates an external data structure (such as microframework authors) may
	1207	argue that the set of circumstances I outlined above is anomalous and
	1208	contrived. They will argue that it just will never happen. If you never
	1209	intend your application to grow beyond one or two or three modules, that's
	1210	probably true. However, as your codebase grows, and becomes spread across a
	1211	greater number of modules, the circumstances in which module-scope code will
	1212	be executed multiple times will become more and more likely to occur and less
	1213	and less predictable. It's not responsible to claim that double-execution of
	1214	module-scope code will never happen. It will; it's just a matter of luck,
	1215	time, and application complexity.
7b511a	1216
8cdb1b	1217	If microframework authors do admit that the circumstance isn't contrived,
96d456	1218	they might then argue that real damage will never happen as the result of the
46f74d	1219	double-execution (or triple-execution, etc.) of module scope code. You would
96d456	1220	be wise to disbelieve this assertion. The potential outcomes of multiple
CM	1221	execution are too numerous to predict because they involve delicate
8cdb1b	1222	relationships between application and framework code as well as chronology of
CM	1223	code execution. It's literally impossible for a framework author to know
	1224	what will happen in all circumstances. But even if given the gift of
	1225	omniscience for some limited set of circumstances, the framework author
	1226	almost certainly does not have the double-execution anomaly in mind when
46f74d	1227	coding new features. They're thinking of adding a feature, not protecting
8cdb1b	1228	against problems that might be caused by the 1% multiple execution case.
CM	1229	However, any 1% case may cause 50% of your pain on a project, so it'd be nice
46f74d	1230	if it never occurred.
7b511a	1231
46f74d	1232	Responsible microframeworks actually offer a back-door way around the problem.
SP	1233	They allow you to disuse decorator-based configuration entirely. Instead of
	1234	requiring you to do the following:
7b511a	1235
CM	1236	.. code-block:: python
16cd50	1237	:linenos:
7b511a	1238
CM	1239	gh = Groundhog('myapp', 'seekrit')
	1240
	1241	@gh.route('/foo/')
	1242	def foo():
	1243	return 'foo'
	1244
	1245	if __name__ == '__main__':
	1246	gh.run()
	1247
46f74d	1248	They allow you to disuse the decorator syntax and go almost all-imperative:
7b511a	1249
CM	1250	.. code-block:: python
16cd50	1251	:linenos:
7b511a	1252
CM	1253	def foo():
	1254	return 'foo'
	1255
	1256	gh = Groundhog('myapp', 'seekrit')
	1257
	1258	if __name__ == '__main__':
	1259	gh.add_route(foo, '/foo/')
	1260	gh.run()
	1261
edd915	1262	This is a generic mode of operation that is encouraged in the Pyramid
8cdb1b	1263	documentation. Some existing microframeworks (Flask, in particular) allow for
CM	1264	it as well. None (other than Pyramid) encourage it. If you never expect
	1265	your application to grow beyond two or three or four or ten modules, it
	1266	probably doesn't matter very much which mode you use. If your application
	1267	grows large, however, imperative configuration can provide better
	1268	predictability.
7b511a	1269
CM	1270	.. note::
	1271
46f74d	1272	Astute readers may notice that Pyramid has configuration decorators too. Aha!
SP	1273	Don't these decorators have the same problems? No. These decorators do not
	1274	populate an external Python module when they are executed. They only mutate
	1275	the functions (and classes and methods) to which they're attached. These
	1276	mutations must later be found during a scan process that has a predictable
	1277	and structured import phase. Module-localized mutation is actually the
	1278	best-case circumstance for double-imports. If a module only mutates itself
	1279	and its contents at import time, if it is imported twice, that's OK, because
	1280	each decorator invocation will always be mutating an independent copy of the
	1281	object to which it's attached, not a shared resource like a registry in
	1282	another module. This has the effect that double-registrations will never be
	1283	performed.
7b511a	1284
d559af	1285
PE	1286	.. _routes_need_ordering:
	1287
3f6547	1288	Routes need relative ordering
0a585a	1289	+++++++++++++++++++++++++++++
7b511a	1290
CM	1291	Consider the following simple `Groundhog
e9bf10	1292	<https://github.com/Pylons/groundhog>`_ application:
7b511a	1293
CM	1294	.. code-block:: python
16cd50	1295	:linenos:
7b511a	1296
CM	1297	from groundhog import Groundhog
	1298	app = Groundhog('myapp', 'seekrit')
	1299
f7d515	1300	@app.route('/admin')
7b511a	1301	def admin():
CM	1302	return '<html>admin page</html>'
	1303
f7d515	1304	@app.route('/:action')
MM	1305	def do_action(action):
7b511a	1306	if action == 'add':
7be3b1	1307	return '<html>add</html>'
7b511a	1308	if action == 'delete':
7be3b1	1309	return '<html>delete</html>'
7b511a	1310	return app.abort(404)
CM	1311
	1312	if __name__ == '__main__':
	1313	app.run()
	1314
96d456	1315	If you run this application and visit the URL ``/admin``, you will see the
3f6547	1316	"admin" page. This is the intended result. However, what if you rearrange the
SP	1317	order of the function definitions in the file?
7b511a	1318
CM	1319	.. code-block:: python
16cd50	1320	:linenos:
7b511a	1321
CM	1322	from groundhog import Groundhog
	1323	app = Groundhog('myapp', 'seekrit')
	1324
f7d515	1325	@app.route('/:action')
MM	1326	def do_action(action):
7b511a	1327	if action == 'add':
7be3b1	1328	return '<html>add</html>'
7b511a	1329	if action == 'delete':
7be3b1	1330	return '<html>delete</html>'
7b511a	1331	return app.abort(404)
CM	1332
f7d515	1333	@app.route('/admin')
7b511a	1334	def admin():
CM	1335	return '<html>admin page</html>'
	1336
	1337	if __name__ == '__main__':
	1338	app.run()
	1339
3f6547	1340	If you run this application and visit the URL ``/admin``, your app will now
SP	1341	return a 404 error. This is probably not what you intended. The reason you see
	1342	a 404 error when you rearrange function definition ordering is that routing
	1343	declarations expressed via our microframework's routing decorators have an
	1344	ordering, and that ordering matters.
7b511a	1345
8cdb1b	1346	In the first case, where we achieved the expected result, we first added a
CM	1347	route with the pattern ``/admin``, then we added a route with the pattern
	1348	``/:action`` by virtue of adding routing patterns via decorators at module
	1349	scope. When a request with a ``PATH_INFO`` of ``/admin`` enters our
	1350	application, the web framework loops over each of our application's route
	1351	patterns in the order in which they were defined in our module. As a result,
3f6547	1352	the view associated with the ``/admin`` routing pattern will be invoked because
SP	1353	it matches first. All is right with the world.
7b511a	1354
8cdb1b	1355	In the second case, where we did not achieve the expected result, we first
CM	1356	added a route with the pattern ``/:action``, then we added a route with the
	1357	pattern ``/admin``. When a request with a ``PATH_INFO`` of ``/admin`` enters
	1358	our application, the web framework loops over each of our application's route
	1359	patterns in the order in which they were defined in our module. As a result,
3f6547	1360	the view associated with the ``/:action`` routing pattern will be invoked
SP	1361	because it matches first. A 404 error is raised. This is not what we wanted; it
	1362	just happened due to the order in which we defined our view functions.
7b511a	1363
3f6547	1364	This is because Groundhog routes are added to the routing map in import order,
SP	1365	and matched in the same order when a request comes in. Bottle, like Groundhog,
	1366	as of this writing, matches routes in the order in which they're defined at
	1367	Python execution time. Flask, on the other hand, does not order route matching
	1368	based on import order. Instead it reorders the routes you add to your
	1369	application based on their "complexity". Other microframeworks have varying
0a585a	1370	strategies to do route ordering.
7b511a	1371
0a585a	1372	Your application may be small enough where route ordering will never cause an
3f6547	1373	issue. If your application becomes large enough, however, being able to specify
SP	1374	or predict that ordering as your application grows larger will be difficult.
	1375	At some point, you will likely need to start controlling route ordering more
	1376	explicitly, especially in applications that require extensibility.
0a585a	1377
96d456	1378	If your microframework orders route matching based on complexity, you'll need
CM	1379	to understand what is meant by "complexity", and you'll need to attempt to
3f6547	1380	inject a "less complex" route to have it get matched before any "more complex"
SP	1381	one to ensure that it's tried first.
0a585a	1382
CM	1383	If your microframework orders its route matching based on relative
	1384	import/execution of function decorator definitions, you will need to ensure
3f6547	1385	that you execute all of these statements in the "right" order, and you'll need
SP	1386	to be cognizant of this import/execution ordering as you grow your application
	1387	or try to extend it. This is a difficult invariant to maintain for all but the
	1388	smallest applications.
0a585a	1389
3f6547	1390	In either case, your application must import the non-``__main__`` modules which
SP	1391	contain configuration decorations somehow for their configuration to be
	1392	executed. Does that make you a little uncomfortable? It should, because
8cdb1b	1393	:ref:`you_dont_own_modulescope`.
7b511a	1394
0428ed	1395	Pyramid uses neither decorator import time ordering nor does it attempt to
3f6547	1396	divine the relative complexity of one route to another as a means to define a
SP	1397	route match ordering. In Pyramid, you have to maintain relative route ordering
	1398	imperatively via the chronology of multiple executions of the
	1399	:meth:`pyramid.config.Configurator.add_route` method. The order in which you
0428ed	1400	repeatedly call ``add_route`` becomes the order of route matching.
CM	1401
	1402	If needing to maintain this imperative ordering truly bugs you, you can use
3f6547	1403	:term:`traversal` instead of route matching, which is a completely declarative
SP	1404	(and completely predictable) mechanism to map code to URLs. While URL dispatch
	1405	is easier to understand for small non-extensible applications, traversal is a
	1406	great fit for very large applications and applications that need to be
	1407	arbitrarily extensible.
	1408
1d1975	1409
288b46	1410	.. _thread_local_nuisance:
SP	1411
	1412	"Stacked object proxies" are too clever / thread locals are a nuisance
7b511a	1413	++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
CM	1414
96d456	1415	Some microframeworks use the ``import`` statement to get a handle to an
CM	1416	object which is not logically global:
7b511a	1417
CM	1418	.. code-block:: python
16cd50	1419	:linenos:
7b511a	1420
CM	1421	from flask import request
	1422
	1423	@app.route('/login', methods=['POST', 'GET'])
	1424	def login():
	1425	error = None
	1426	if request.method == 'POST':
	1427	if valid_login(request.form['username'],
	1428	request.form['password']):
	1429	return log_the_user_in(request.form['username'])
	1430	else:
	1431	error = 'Invalid username/password'
	1432	# this is executed if the request method was GET or the
	1433	# credentials were invalid
	1434
2b5025	1435	The `Pylons 1.X
19d341	1436	<https://docs.pylonsproject.org/projects/pylons-webframework/en/latest/>`_
1cb30e	1437	web framework uses a similar strategy. It calls these things "Stacked Object
SP	1438	Proxies", so, for purposes of this discussion, I'll do so as well.
7b511a	1439
8cdb1b	1440	Import statements in Python (``import foo``, ``from bar import baz``) are
CM	1441	most frequently performed to obtain a reference to an object defined globally
96d456	1442	within an external Python module. However, in normal programs, they are
8cdb1b	1443	never used to obtain a reference to an object that has a lifetime measured by
CM	1444	the scope of the body of a function. It would be absurd to try to import,
	1445	for example, a variable named ``i`` representing a loop counter defined in
	1446	the body of a function. For example, we'd never try to import ``i`` from the
7b511a	1447	code below:
CM	1448
	1449	.. code-block:: python
16cd50	1450	:linenos:
7b511a	1451
CM	1452	def afunc():
	1453	for i in range(10):
edfc4f	1454	print(i)
7b511a	1455
288b46	1456	By its nature, the request object that is created as the result of a WSGI
SP	1457	server's call into a long-lived web framework cannot be global, because the
	1458	lifetime of a single request will be much shorter than the lifetime of the
	1459	process running the framework. A request object created by a web framework
	1460	actually has more similarity to the ``i`` loop counter in our example above
	1461	than it has to any comparable importable object defined in the Python standard
96d456	1462	library or in normal library code.
7b511a	1463
8cdb1b	1464	However, systems which use stacked object proxies promote locally scoped
288b46	1465	objects, such as ``request``, out to module scope, for the purpose of being
96d456	1466	able to offer users a nice spelling involving ``import``. They, for what I
288b46	1467	consider dubious reasons, would rather present to their users the canonical way
SP	1468	of getting at a ``request`` as ``from framework import request`` instead of a
	1469	saner ``from myframework.threadlocals import get_request; request =
	1470	get_request()``, even though the latter is more explicit.
7b511a	1471
8cdb1b	1472	It would be most explicit if the microframeworks did not use thread local
288b46	1473	variables at all. Pyramid view functions are passed a request object. Many of
SP	1474	Pyramid's APIs require that an explicit request object be passed to them. It is
	1475	possible to retrieve the current Pyramid request as a threadlocal variable,
	1476	but it is an "in case of emergency, break glass" type of activity. This
	1477	explicitness makes Pyramid view functions more easily unit testable, as you
	1478	don't need to rely on the framework to manufacture suitable "dummy" request
	1479	(and other similarly-scoped) objects during test setup. It also makes them
	1480	more likely to work on arbitrary systems, such as async servers, that do no
	1481	monkeypatching.
	1482
7b511a	1483
b0fcf1	1484	.. _explicitly_wsgi:
SP	1485
7b511a	1486	Explicitly WSGI
CM	1487	+++++++++++++++
	1488
8cdb1b	1489	Some microframeworks offer a ``run()`` method of an application object that
CM	1490	executes a default server configuration for easy execution.
7b511a	1491
8cdb1b	1492	Pyramid doesn't currently try to hide the fact that its router is a WSGI
CM	1493	application behind a convenience ``run()`` API. It just tells people to
	1494	import a WSGI server and use it to serve up their Pyramid application as per
	1495	the documentation of that WSGI server.
7b511a	1496
8cdb1b	1497	The extra lines saved by abstracting away the serving step behind ``run()``
b0fcf1	1498	seems to have driven dubious second-order decisions related to its API in some
SP	1499	microframeworks. For example, Bottle contains a ``ServerAdapter`` subclass for
	1500	each type of WSGI server it supports via its ``app.run()`` mechanism. This
	1501	means that there exists code in ``bottle.py`` that depends on the following
	1502	modules: ``wsgiref``, ``flup``, ``paste``, ``cherrypy``, ``fapws``,
8cdb1b	1503	``tornado``, ``google.appengine``, ``twisted.web``, ``diesel``, ``gevent``,
b0fcf1	1504	``gunicorn``, ``eventlet``, and ``rocket``. You choose the kind of server you
SP	1505	want to run by passing its name into the ``run`` method. In theory, this sounds
	1506	great: I can try out Bottle on ``gunicorn`` just by passing in a name! However,
	1507	to fully test Bottle, all of these third-party systems must be installed and
	1508	functional. The Bottle developers must monitor changes to each of these
	1509	packages and make sure their code still interfaces properly with them. This
	1510	increases the number of packages required for testing greatly; this is a lot
	1511	of requirements. It is likely difficult to fully automate these tests due to
	1512	requirements conflicts and build issues.
7b511a	1513
8cdb1b	1514	As a result, for single-file apps, we currently don't bother to offer a
b0fcf1	1515	``run()`` shortcut. We tell folks to import their WSGI server of choice and run
SP	1516	it by hand. For the people who want a server abstraction layer, we suggest that
	1517	they use PasteDeploy. In PasteDeploy-based systems, the onus for making sure
	1518	that the server can interface with a WSGI application is placed on the server
	1519	developer, not the web framework developer, making it more likely to be timely
	1520	and correct.
7b511a	1521
583cae	1522	Wrapping up
7b511a	1523	+++++++++++
CM	1524
583cae	1525	Here's a diagrammed version of the simplest pyramid application, where the
SP	1526	inlined comments take into account what we've discussed in the
7b511a	1527	:ref:`microframeworks_smaller_hello_world` section.
CM	1528
	1529	.. code-block:: python
4ef406	1530	:linenos:
7b511a	1531
4ef406	1532	from wsgiref.simple_server import make_server # explicitly WSGI
SP	1533	from pyramid.config import Configurator # to configure app registry
	1534	from pyramid.response import Response # explicit response, no threadlocal
7b511a	1535
4ef406	1536	def hello_world(request): # accept a request; no request threadlocal reqd
SP	1537	# explicit response object means no response threadlocal
	1538	return Response('Hello world!')
7b511a	1539
4ef406	1540	if __name__ == '__main__':
SP	1541	with Configurator() as config: # no global application object
	1542	config.add_view(hello_world) # explicit non-decorator registration
	1543	app = config.make_wsgi_app() # explicitly WSGI
	1544	server = make_server('0.0.0.0', 8080, app)
	1545	server.serve_forever() # explicitly WSGI
7b511a	1546
583cae	1547
6c36d7	1548	Pyramid doesn't offer pluggable apps
82e3e3	1549	------------------------------------
CM	1550
	1551	It is "Pyramidic" to compose multiple external sources into the same
600554	1552	configuration using :meth:`~pyramid.config.Configurator.include`. Any
98b28d	1553	number of includes can be done to compose an application; includes can even
CM	1554	be done from within other includes. Any directive can be used within an
	1555	include that can be used outside of one (such as
6c36d7	1556	:meth:`~pyramid.config.Configurator.add_view`).
82e3e3	1557
96d456	1558	Pyramid has a conflict detection system that will throw an error if two
CM	1559	included externals try to add the same configuration in a conflicting way
	1560	(such as both externals trying to add a route using the same name, or both
	1561	externals trying to add a view with the same set of predicates). It's awful
	1562	tempting to call this set of features something that can be used to compose a
	1563	system out of "pluggable applications". But in reality, there are a number
	1564	of problems with claiming this:
82e3e3	1565
CM	1566	- The terminology is strained. Pyramid really has no notion of a
	1567	plurality of "applications", just a way to compose configuration
	1568	from multiple sources to create a single WSGI application. That
	1569	WSGI application may gain behavior by including or disincluding
	1570	configuration, but once it's all composed together, Pyramid
	1571	doesn't really provide any machinery which can be used to demarcate
	1572	the boundaries of one "application" (in the sense of configuration
	1573	from an external that adds routes, views, etc) from another.
	1574
96d456	1575	- Pyramid doesn't provide enough "rails" to make it possible to integrate
CM	1576	truly honest-to-god, download-an-app-from-a-random-place
	1577	and-plug-it-in-to-create-a-system "pluggable" applications. Because
	1578	Pyramid itself isn't opinionated (it doesn't mandate a particular kind of
	1579	database, it offers multiple ways to map URLs to code, etc), it's unlikely
	1580	that someone who creates something application-like will be able to
	1581	casually redistribute it to J. Random Pyramid User and have it just work by
	1582	asking him to config.include a function from the package. This is
	1583	particularly true of very high level components such as blogs, wikis,
	1584	twitter clones, commenting systems, etc. The integrator (the Pyramid
	1585	developer who has downloaded a package advertised as a "pluggable app")
	1586	will almost certainly have made different choices about e.g. what type of
	1587	persistence system he's using, and for the integrator to appease the
	1588	requirements of the "pluggable application", he may be required to set up a
	1589	different database, make changes to his own code to prevent his application
	1590	from shadowing the pluggable app (or vice versa), and any other number of
	1591	arbitrary changes.
82e3e3	1592
CM	1593	For this reason, we claim that Pyramid has "extensible" applications,
	1594	not pluggable applications. Any Pyramid application can be extended
	1595	without forking it as long as its configuration statements have been
96d456	1596	composed into things that can be pulled in via ``config.include``.
82e3e3	1597
CM	1598	It's also perfectly reasonable for a single developer or team to create a set
	1599	of interoperating components which can be enabled or disabled by using
	1600	config.include. That developer or team will be able to provide the "rails"
	1601	(by way of making high-level choices about the technology used to create the
	1602	project, so there won't be any issues with plugging all of the components
	1603	together. The problem only rears its head when the components need to be
	1604	distributed to arbitrary users. Note that Django has a similar problem
	1605	with "pluggable applications" that need to work for arbitrary third parties,
	1606	even though they provide many, many more rails than does Pyramid. Even the
	1607	rails they provide are not enough to make the "pluggable application" story
	1608	really work without local modification.
	1609
	1610	Truly pluggable applications need to be created at a much higher level than a
	1611	web framework, as no web framework can offer enough constraints to really
96d456	1612	make them work out of the box. They really need to plug into an application,
CM	1613	instead. It would be a noble goal to build an application with Pyramid that
	1614	provides these constraints and which truly does offer a way to plug in
	1615	applications (Joomla, Plone, Drupal come to mind).
82e3e3	1616
8cdb1b	1617	Pyramid Has Zope Things In It, So It's Too Complex
CM	1618	--------------------------------------------------
	1619
	1620	On occasion, someone will feel compelled to post a mailing list message that
	1621	reads something like this:
	1622
	1623	.. code-block:: text
	1624
4ef406	1625	had a quick look at pyramid ... too complex to me and not really
SP	1626	understand for which benefits.. I feel should consider whether it's time
	1627	for me to step back to django .. I always hated zope (useless ?)
	1628	complexity and I love simple way of thinking
8cdb1b	1629
CM	1630	(Paraphrased from a real email, actually.)
	1631
283494	1632	Let's take this criticism point-by-point.
8cdb1b	1633
CM	1634	Too Complex
	1635	+++++++++++
	1636
4269f3	1637	If you can understand this "hello world" program, you can use Pyramid:
8cdb1b	1638
CM	1639	.. code-block:: python
4ef406	1640	:linenos:
8cdb1b	1641
4ef406	1642	from wsgiref.simple_server import make_server
SP	1643	from pyramid.config import Configurator
	1644	from pyramid.response import Response
8cdb1b	1645
4ef406	1646	def hello_world(request):
4b84f6	1647	return Response('Hello World!')
8cdb1b	1648
4ef406	1649	if __name__ == '__main__':
SP	1650	with Configurator() as config:
4b84f6	1651	config.add_route('hello', '/')
SP	1652	config.add_view(hello_world, route_name='hello')
4ef406	1653	app = config.make_wsgi_app()
4b84f6	1654	server = make_server('0.0.0.0', 6543, app)
4ef406	1655	server.serve_forever()
8cdb1b	1656
b8dd44	1657	Pyramid has over 1200 pages of documentation (printed), covering topics from
8eb4e1	1658	the very basic to the most advanced. Nothing is left undocumented, quite
adfcf6	1659	literally. It also has an awesome, very helpful community. Visit the
8eb4e1	1660	`#pyramid IRC channel on freenode.net
SP	1661	<https://webchat.freenode.net/?channels=pyramid>`_ and see.
8cdb1b	1662
CM	1663	Hate Zope
	1664	+++++++++
	1665
	1666	I'm sorry you feel that way. The Zope brand has certainly taken its share of
	1667	lumps over the years, and has a reputation for being insular and mysterious.
	1668	But the word "Zope" is literally quite meaningless without qualification.
	1669	What part of Zope do you hate? "Zope" is a brand, not a technology.
	1670
	1671	If it's Zope2-the-web-framework, Pyramid is not that. The primary designers
	1672	and developers of Pyramid, if anyone, should know. We wrote Pyramid's
	1673	predecessor (:mod:`repoze.bfg`), in part, because we knew that Zope 2 had
	1674	usability issues and limitations. :mod:`repoze.bfg` (and now :app:`Pyramid`)
	1675	was written to address these issues.
	1676
	1677	If it's Zope3-the-web-framework, Pyramid is definitely not that. Making
	1678	use of lots of Zope 3 technologies is territory already staked out by the
	1679	:term:`Grok` project. Save for the obvious fact that they're both web
125ea4	1680	frameworks, :app:`Pyramid` is very, very different than Grok. Grok exposes
8cdb1b	1681	lots of Zope technologies to end users. On the other hand, if you need to
CM	1682	understand a Zope-only concept while using Pyramid, then we've failed on some
	1683	very basic axis.
	1684
	1685	If it's just the word Zope: this can only be guilt by association. Because a
	1686	piece of software internally uses some package named ``zope.foo``, it doesn't
	1687	turn the piece of software that uses it into "Zope". There is a lot of
	1688	great software written that has the word Zope in its name. Zope is not
	1689	some sort of monolithic thing, and a lot of its software is usable
	1690	externally. And while it's not really the job of this document to defend it,
	1691	Zope has been around for over 10 years and has an incredibly large, active
	1692	community. If you don't believe this,
9ce94f	1693	https://pypi.org/search/?q=zope is an eye-opening reality
8cdb1b	1694	check.
CM	1695
	1696	Love Simplicity
	1697	+++++++++++++++
	1698
	1699	Years of effort have gone into honing this package and its documentation to
	1700	make it as simple as humanly possible for developers to use. Everything is a
	1701	tradeoff, of course, and people have their own ideas about what "simple" is.
	1702	You may have a style difference if you believe Pyramid is complex. Its
	1703	developers obviously disagree.
	1704
fa2e24	1705	Other Challenges
CM	1706	----------------
fbfea7	1707
809744	1708	Other challenges are encouraged to be sent to the `Pylons-devel
1cb30e	1709	<https://groups.google.com/forum/#!forum/pylons-devel>`_ maillist. We'll try
SP	1710	to address them by considering a design change, or at very least via exposition
	1711	here.