Internet-Draft | CDNI Metadata Model Extensions | March 2022 |
Goldstein, et al. | Expires 8 September 2022 | [Page] |
The Content Delivery Network Interconnection (CDNI) Metadata interface enables interconnected Content Delivery Networks (CDNs) to exchange content distribution metadata in order to enable content acquisition and delivery. To facilitate a wider set of use cases such as Open Caching, this document describes extensions to the CDNI Metadata object model and its associated Capabilities model as documented in "CDNI Metadata" RFC8006 and "CDNI Request Routing: Footprint and Capabilities Semantics" RFC8008 .¶
This document is a reflection of the content in the Streaming Video Alliance specification titled "SVA Configuration Interface: Part 2 Extensions to CDNI Metadata Object Model".¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 8 September 2022.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
The Content Delivery Network Interconnection (CDNI) Metadata interface enables interconnected Content Delivery Networks (CDNs) to exchange content distribution metadata in order to enable content acquisition and delivery. To facilitate a wider set of use cases encountered in the commercial CDN and Open Caching ecosystems, this document describes extensions to the CDNI Metadata object model and its associated Capabilities model.¶
The objectives of this document are:¶
Scope this document ADDRESSES:¶
Scope this document DOES NOT ADDRESS:¶
For consistency with other CDNI documents this document follows the CDNI convention of uCDN (upstream CDN) and dCDN (downstream CDN). It should be noted, however, that uCDN and dCDN are roles that can be played by a variety of entities in the distribution ecosystem. A Content Provider, for example, can play the roles of a uCDN, while a commercial CDN or Open Caching system can play either the roles of a uCDN or dCDN. Additionally, this document reuses the terminology defined in [RFC6707],[RFC7336], [RFC8006], [RFC8007] and [RFC8804].¶
The following terms are used throughout this document:¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
This section details extensions to the CDNI Metadata model as defined in Section 4 of [RFC8006], expressed as a set of new GenericMetadata objects. To preserve backward compatibility with [RFC8006], no changes are proposed to the original set of GenericMetadata.¶
The remainder of this section presents the extended set of GenericMetadata objects organized by the categories in the above diagram.¶
Note: In the following sections, the term "mandatory-to-specify" is used to convey which properties MUST be included when serializing a given capability object. When mandatory-to-specify is defined as "Yes" for an individual property, it means that if the object containing that property is included in a message, then the mandatory-to-specify property MUST also be included.¶
In addition to the cache control policies currently specified by CDNI metadata, content providers often need more fine-grained control over CDN caching, including scenarios where it is desirable to override or adjust cache-control headers from the origin.¶
The following additional capabilities are needed for general CDN and open caching use cases:¶
CachePolicy is a new GenericMetadata object that allows for the uCDN to specify internal caching policies for the dCDN, as well as external caching policies advertised to clients of the dCDN (overriding any cache control policy set in the response from the uCDN).¶
Property: internal¶
Property: external¶
Property: force¶
Example 1: An MI.CachePolicy that sets the internal cache control policy to five seconds. The external cache policy is set to 'no-cache':¶
{ "generic-metadata-type": "MI.CachePolicy", "generic-metadata-value": { "internal": "5", "external": "no-cache", "force": "true" } }¶
Example 2: An MI.CachePolicy that sets the internal cache control policy to "as-is" (keep the policy set in the response from the uCDN). The external cache policy is set to 'no-cache:¶
{ "generic-metadata-type": "MI.CachePolicy", "generic-metadata-value": { "internal": "as-is", "external": "no-cache", "force": "true" } }¶
Example 3: An MI.CachePolicy in the context of the processing stages model that sets a caching policy only if the HTTP status code received from the origin is a 200. In this example, the internal cache control policy is set to five seconds. The external cache policy is set to 'no-cache'. Force is set to 'False', indicating that the MI.CachePolicy only applies if there is no cache policy in the response from the uCDN.¶
{ "generic-metadata-type": "MI.ProcessingStages", "generic-metadata-value": { "origin-response": [ { "match": { "expression": "resp.status == 200" }, "stage-metadata": { "generic-metadata": [ { "generic-metadata-type": "MI.CachePolicy", "generic-metadata-value": { "internal": "5", "external": "no-cache", "force": "false" } } ] } } ] } }¶
NegativeCachePolicy is a new GenericMetadata object that allows for the specification of caching policies based on error response codes received from the origin.¶
Property: error-codes¶
Property: cache-policy¶
Example: A MI.NegativeCachePolicy that applies to HTTP error codes: "404", "503", "504" and sets the internal cache control policy to five seconds and external to 'no-cache'.¶
{ "generic-metadata-type": "MI.NegativeCachePolicy", "generic-metadata-value": { "error-codes": [ "404", "503", "504" ], "cache-policy": { "internal": "5", "external": "no-cache", "force": "true" } } }¶
MI.StaleContentCachePolicy is a new GenericMetadata object that allows the uCDN to specify the policy to use by a dCDN when responding with stale content. For example, this policy allows the content provider to specify that stale content be served from cache for a specified time period while refreshes from the origin occur asynchronously.¶
Property: stale-while-revalidating¶
Property: stale-if-error¶
Property: failed-refresh-ttl¶
Example 1: A MI.StaleContentCachePolicy where stale-while-revalidating is true, instructing the dCDN to respond with a stale cached version of the resource while it refreshes the resource with the uCDN in the background:¶
{ "generic-metadata-type": "MI.StaleContentCachePolicy", "generic-metadata-value": { "stale-while-revalidating": true } }¶
Example 2: A MI.StaleContentCachePolicy where stale-if-error instructs the dCDN to use the stale cached resource if it receives an error of type 503 or 504 when trying to refresh the resource with the uCDN.¶
failed-refresh-ttl instructs the dCDN to use a five second cache TTL on the resource that receives an error when refreshing from the uCDN. That is, after five seconds, the dCDN will attempt to refresh the resource with the uCDN.¶
{ "generic-metadata-type": "MI.StaleContentCachePolicy", "generic-metadata-value": { "stale-if-error": [ "503", "504" ], "failed-refresh-ttl": "5" } }¶
Example 3: A MI.StaleContentCachePolicy where stale-while-revalidating is true, instructing the dCDN to respond with a stale cached version of the resource while it refreshes the resource with the uCDN in the background.¶
stale-if-error instructs the dCDN to use the stale cached resource if it receives an error of type 404, 503, or 504 when trying to refresh the resource with the uCDN.¶
failed-refresh-ttl instructs the dCDN to use a five second cache TTL on the resource that receives an error when refreshing from the uCDN. That is, after five seconds, the dCDN will attempt to refresh the resource with the uCDN.¶
{ "generic-metadata-type": "MI.StaleContentCachePolicy", "generic-metadata-value": { "stale-while-revalidating": "true", "stale-if-error": [ "404", "503", "504" ], "failed-refresh-ttl": "5" } }¶
CacheBypassPolicy is a new GenericMetadata object that allows a client request to be set as non-cacheable. It is expected that this feature will be used to allow clients to bypass cache when testing the uCDN fill path. Note: CacheBypassPolicy is typically used in conjunction with a path match or match expression on a header value or query parameter. Any content previously cached (by client requests that do not set CacheBypassPolicy) is not evicted.¶
Property: bypass-cache¶
Example 1: A MI.CacheBypassPolicy with the client HTTP header of: CDN-BYPASS: "True":¶
{ "generic-metadata-type": "MI.ProcessingStages", "generic-metadata-value": { "client-request": [ { "match": { "expression": "req.h.cdn-bypass == 'true'" }, "stage-metadata": { "generic-metadata": [ { "generic-metadata-type": "MI.CacheBypassPolicy", "generic-metadata-value": { "bypass-cache": "true" } } ] } } ] } }¶
Example 2: A MI.CacheBypassPolicy that applies to all requests where the host header is bypass.example.com:¶
{ "generic-metadata-type": "MI.ProcessingStages", "generic-metadata-value": { "client-request": [ { "match": { "expression": "req.h.host == 'bypass.example.com'" }, "stage-metadata": { "generic-metadata": [ { "generic-metadata-type": "MI.CacheBypassPolicy", "generic-metadata-value": { "bypass-cache": "true" } } ] } } ] } }¶
While the properties provided by the standard CDNi metadata Cache object (See Section 4.2.6 of [RFC8006]) provide some simple control over the construction of the cache key, it is typical in advanced CDN configurations to generate cache keys that are dynamically constructed via lightweight processing of various properties of the HTTP request and/or response. As an example, an origin may specify a cache key as a value returned in a specific HTTP response header.¶
ComputedCacheKey is a new GenericMetadata object that allows for the specification of a cache key using the metadata expression language. Typical use cases would involve the construction of a cache key from one or more elements of the HTTP request. In cases where both the ComputedCacheKey and the Cache object are applied, the ComputedCacheKey will take precedence.¶
Property: expression¶
Example, using a custom request header as the cache key instead of the URI path:¶
{ "generic-metadata-type": "MI.ComputedCacheKey", "generic-metadata-value": { "expression": "req.h.X-Cache-Key" } }¶
The CDNI metadata definitions for sources (also known as origins in the CDN industry), are extended to provide the following capabilities required:¶
SourceMetadataExtended is an alternative to the CDNI standard SourceMetadata object, which adds a property to specify load balancing across multiple sources, as well as a SourceExtended sub-object with additional attributes to the CDNI standard Source object. While both SourceMetadataExtended and SourceMetadata can be provided for backward compatibility, a dCDN that advertises capability for SourceMetadataExtended will ignore SourceMetadata if both are provided for a given host or path match.¶
Example of a SourceMetadataExtended object with the associated LoadBalanceMetadata configuration object:¶
{ "generic-metadata-type": "MI.SourceMetadataExtended", "generic-metadata-value": { "sources": [ { "endpoints": [ "a.service123.ucdn.example", "b.service123.ucdn.example" ], "protocol": "http/1.1" }, { "endpoints": [ "origin.service123.example" ], "protocol": "http/1.1" } ], "load-balance": { "balance-algorithm": "content-hash", "balance-path-pattern": "^/prod/(.*)/.*\\.ts$" } } }¶
SourceExtended is an alternative to the CDNI standard Source object with additional metadata. It contains all the attributes of the [RFC8006] Source object (acquisition-auth, endpoints, and protocol), with additions specified below.¶
Property: acquisition-auth¶
Property: endpoints¶
Property: protocol¶
Property: origin-host¶
Property: webroot¶
Property: follow-redirects¶
Property: timeout-ms¶
Property: failover-errors¶
Example of a SourceExtended object that describes a pair of endpoints (servers) that the dCDN can use to acquire content for the applicable host and/or URI path:¶
{ "generic-metadata-type": "MI.SourceMetadataExtended", "generic-metadata-value": { "sources": [ { "endpoints": [ "a.service123.ucdn.example", "b.service123.ucdn.example:8443" ], "protocol": "https/1.1", "origin-host": "internal.example.com", "webroot": "/prod", "follow-redirects": false, "timeout-ms": 4000, "failover-errors": [ "502", "503", "504" ] }, { "endpoints": [ "origin.service123.example" ], "protocol": "http/1.1", "webroot": "/prod", "follow-redirects": true, "timeout-ms": 8000 } ] } }¶
The LoadBalanceMetadata object defines how content acquisition requests are distributed over the SourceExtended objects listed in the SourceMetadataExtended object.¶
Property: balance-algorithm¶
o random: Requests are distributed over the sources in proportion to their associated weights.¶
o content-hash: Requests are distributed over the sources in a consistent fashion, based on the balance-path-pattern property.¶
o ip-hash: Requests are distributed over the sources in a consistent fashion based on the IP address of the client requestor.¶
Property: balance-weights¶
Property: balance-path-pattern¶
Example 1: The LoadBalanceMetadata object distributes content acquisition requests over sources using a content-hash algorithm:¶
{ "generic-metadata-type": "MI.LoadBalanceMetadata", "generic-metadata-value": { "balance-algorithm": "content-hash", "balance-path-pattern": "^/prod/(.*)/.*\\.ts$" } }¶
Example 2: The LoadBalanceMetadata object distributes content acquisition requests over sources using the random algorithm:¶
{ "generic-metadata-type": "MI.LoadBalanceMetadata", "generic-metadata-value": { "balance-algorithm": "random", "balance-weights": [ 1, 2, 1 ] } }¶
To meet the typical industry requirements for authenticating CDNs to external origins, two new authentication types are defined.¶
auth-type: MI.HeaderAuth¶
auth-type: MI.AWSv4Auth¶
The HeaderAuth metadata object is used in the auth-value property of the¶
Auth object, as defined in [RFC8006] section 4.2.7, and may be applied to any¶
source by including or referencing it under its authentication property. This method of authentication provides a simple capability for a mutually agreed upon header to be added by the CDN to all requests sent to a specific origin. Note that if a dynamically generated header value is required, the RequestTransform capabilities within StageProcessing can be used.¶
Example Auth object for header authentication:¶
{ "generic-metadata-type": "MI.SourceMetadataExtended", "generic-metadata-value": { "sources": [ { "endpoints": [ "origin.example.com" ], "protocol": "http/1.1", "acquisition-auth": { "generic-metadata-type": "MI.Auth", "generic-metadata-value": { "auth-type": "MI.HeaderAuth", "auth-value": { "header-name": "X-Origin-Auth", "header-value": "SECRETKEYJKSDHFSIFUI4UFH78HW4NF7" } } } } ] } }¶
The AWSv4Auth metadata object is used in the auth-value property of the Auth object as defined in [RFC8006] section 4.2.7, and may be applied to any source by including or referencing it under its authentication property.¶
AWSv4 authentication causes upstream requests to have a signature applied, following the method described in [AWSv4Method]. A hash-based signature is calculated over the request URI and specified headers, and provided in an Authorization: header on the upstream request. The signature is tied to a pre-shared secret key specific to an AWS service, region, and key ID.¶
Property: access-key-id¶
Property: secret-access-key¶
Property: aws-region¶
Property: aws-service¶
Property: host-name¶
Example Auth object for AWSv4 authentication:¶
{ "generic-metadata-type": "MI.SourceMetadataExtended", "generic-metadata-value": { "sources": [ { "endpoints": [ "origin.example.com" ], "protocol": "http/1.1", "acquisition-auth": { "generic-metadata-type": "MI.Auth", "generic-metadata-value": { "auth-type": "MI.AWSv4Auth", "auth-value": { "access-key-id": "MYACCESSKEYID", "secret-access-key": "SECRETKEYJKSDHFSIUHKWRGHHF", "aws-region": "us-west-1" } } } } ] } }¶
CDNs typically require a set of configuration metadata to inform processing of responses downstream (at the edge and in the user agent). This section specifies GenericMetadata objects to meet those requirements.¶
Delegation of traffic between a CDN over an open caching node based on HTTP redirection does change the domain name in the client requests. This represents a cross-origin request that must be managed appropriately using Cross-Origin Resource Sharing (CORS) headers in the responses.¶
The dynamic generation of CORS headers is typical in modern HTTP request processing and avoids CORS validation forwarded to the CDN origin servers, particularly with the preflight OPTIONS requests. The CDNI metadata model requires extensions to specify how a CDN or open caching node should generate and evaluate these headers.¶
Required capabilities:¶
Simple CORS requests are those where both HTTP method and headers in the request are included in the safe list defined by the World Wide Web Consortium [W3C]. The user agent (UA) request can include an origin header set to the URL domain of the webpage that the player runs. Depending on the metadata configuration, the logic to apply by the open caching node (OCN) is:¶
When a UA makes a request that includes a method or headers that are not included in the safe-list, the client will make a CORS preflight request using the OPTIONS method to the resource including the origin header. If CORS is enabled and the requests passes the origin validation, the OCN SHOULD respond with the set of headers that indicate what is permitted for that resource, including one or more of the following:¶
CrossoriginPolicy is a GenericMetadata object that allows for the specification of dynamically generated CORS headers.¶
Property: allow-origin¶
Property: expose-headers¶
Property: allow-methods¶
Property: allow-headers¶
Property: allow-credentials¶
Property: max-age¶
The MI.AccessControlAllowOrigin object has the following properties:¶
Property: allow-list¶
Property: wildcard-return¶
The examples below demonstrate how to configure response headers dynamically for CORS validation.¶
Example 1: A simple CORS validation configuration:¶
{ "generic-metadata-type": "MI.CrossoriginPolicy", "generic-metadata-value": { "allow-origin": { "allow-list": [ { "pattern": "*" } ], "wildcard-return": true } } }¶
Example 2: Validation of a preflight request when some of the headers included in the subsequent object request are not included in the CORS specification safelist:¶
{ "generic-metadata-type": "MI.CrossoriginPolicy", "generic-metadata-value": { "allow-origin": { "allow-list": [ { "pattern": "*://sourcepage.example.com" }, "wildcard-return": false }, "allow-methods": [ "GET", "POST" ], "allow-credentials": true, "allow-headers": [ "X-PINGOTHER", "Content-Type" ], "expose-headers": [ "X-User", "Authorization" ], "max-age": 3600 } } }¶
Downstream CDNs often have the ability to compress HTTP response bodies in cases where the client has declared that it can accept compressed responses (via an Accept-Encoding header), but the source/origin has returned an uncompressed response.¶
The specific compression algorithm used by the dCDN is negotiated by the client's Accept-Encoding header according to [RFC7694] (including q= preferences) and the compression capabilities available on the dCDN.¶
In addition, HeaderTransform allows the uCDN to normalize, or modify, the Accept-Encoding header to allow for fine-grain control over the selection of the compression algorithm (e.g., gzip, compress, deflate, br, etc.).¶
AllowCompress is a new GenericMetadata object that allows the dCDN to compress content before sending to the client.¶
Property: allow-compress¶
Example 1: An MI.AllowCompress that allows manifests (*.m3u8) to be compressed by the dCDN:¶
{ "match": { "expression": "req.h.uri *= '*.m3u8'" }, "stage-metadata": { "generic-metadata": [ { "generic-metadata-type": "MI.AllowCompress", "generic-metadata-value": { "allow-compress": "true" } } ] } }¶
Example 2: An MI.AllowCompress that allows manifests (*.m3u8) to be compressed by the dCDN but normalizing the client's Accept-Encoding header:¶
{ "match": { "expression": "req.h.accept-encoding *= '*gzip*'" }, "stage-metadata": { "generic-metadata": [ { "generic-metadata-type": "MI.AllowCompress", "generic-metadata-value": { "allow-compress": "true" } } ] } }¶
Content delivery networks often apply different infrastructure, network routes, and internal metadata for different types of traffic. Delivery of large static objects (such as software downloads), may, for example, use different network routes than video stream delivery. In an HTTP adaptive bitrate video service, every video title corresponds to a set of video files and descriptors according to different video protocols, and this is independent of the type of service (Video-on-Demand, Live, Catch-up, etc.).¶
The way the video service is consumed by the user agents can vary. For instance, a segment that belongs to a Video-on-Demand (VoD) title can be requested for every moment the content is available for the user agents to consume, while a segment of live content will be only requested as long as the time-shift duration is configured for that service. Knowing those differences, a CDN or OCN provider can implement specific strategies that will maximize performance and thereby provide more available capacity to the upstream provider. It should be noted that the dCDNs handling of the traffic types is implementation-specific and not prescribed here.¶
TrafficType metadata defines a set of descriptors that characterize either the type or usage of the traffic, enabling CDNs and OCNs to apply any internal configuration rules without exposing an unnecessary number of internal details. Note that the interpretation of these traffic types and application of rules such as rate limiting or delivery pacing are implementation specific.¶
A TrafficType definition example for HostMetadata:¶
{ "generic-metadata-type": "MI.TrafficType", "generic-metadata-value": { "traffic-type": "vod", "hints": [ "low-latency", "catch-up" ] } }¶
Configuration metadata is required to permit several levels of OCN selection policies. For example, in a mobile network, several physical locations are possible (i.e., candidates) for hosting the OCN that will take charge in the delegation for the uCDN. This is the case when the cache is virtualized and deployed dynamically. Depending on the OCN selection policy (which may be a cost driver), the dCDN may attempt to favor certain types of caches at the edge, for example. The default OCN selection policy might be "best-effort". Another one might be linked to the network characteristics like "Edge" or ("average latency< 10ms").¶
OcnSelection is a new GenericMetadata object that allows the uCDN to indicate to the dCDN a preference in terms of OCN selection.¶
Property: ocn-delivery¶
Property: ocn-type¶
Property: ocn-selection¶
An ocn-delivery object contains the following properties:¶
It is typical in CDN configurations to define matching rules and metadata that are to be applied at specific stages in the request processing pipeline. For example, it may be required to append a host header prior to forwarding a request to an origin, or modify the response returned from an origin prior to storing in the cache.¶
Processing stages:¶
Requirements:¶
A ProcessingStages object is a new GenericMetadata which describes the matching rules, metadata, and transformations to be applied at specific stages in the request processing pipeline. The processing rules and transformations are defined as a child data model referenced within a ProcessingStages object, as defined below.¶
Each of the four processing stages is represented by an array of StageRules objects, with each StageRules object defining criteria along with metadata that MUST be applied if the match applies to "True". It should be noted that the StageRules objects in the array are evaluated and processed in order. A possible future extension to this processing model could allow for an if-else structure, where processing for a stage is halted upon the first match of a StageRules expression.¶
Property: client-request¶
Property: origin-request¶
Property: origin-response¶
Property: client-response¶
Example specifying all four processing stages. In this example, the client-request stage has two StageRules, appling one set of metadata if "ExpressionMatch1" evaluates to "True" and applying another set of metadata if "ExpressionMatch2" evaluates to "True".¶
{ "generic-metadata-type": "MI.ProcessingStages", "generic-metadata-value": { "client-request" : [ { "match" : < ExpressionMatch1 for conditional metadata > "stage-metadata" : <StageMetadata1 for clientRequest stage>, }, { "match" : <Additional ExpressionMatch2 for conditional metadata > "stage-metadata" : <StageMetadata2 for clientRequest stage>, } ], "origin-request" : [{ "match" : <Optional ExpressionMatch for conditional metadata > "stage-metadata" : <StageMetadata for originRequest stage>, }], "origin-response" : [{ "match" : <Optional ExpressionMatch for conditional metadata > "stage-metadata" : <StageMetadata for originResponse stage>, }], "client-response" : [{ "match" : <Optional ExpressionMatch for conditional metadata > "stage-metadata" : <StageMetadata for clientResponse stage>, }] } }¶
A StageRules object is used within the context of ProcessingStages to define elements in a list of match rules and stage-specific metadata and transformations that MUST be applied conditionally on a rich expression match.¶
Property: match¶
Property: stage-metadata¶
An example of StageRules that are applied just after responses are received from the origin. In this example, receipt of a response status code of 304 from the origin indicates that CachePolicy metadata SHOULD be applied (as specified via an external HREF), and that response headers SHOULD be modified (X-custom-response-header added and ETag deleted).¶
{ "match": { "expression": "resp.status == 304" }, "stage-metadata": { "generic-metadata": [ { "type": "MI.CachePolicy", "href": "https://metadata.ucdn.example/origin_response_cache" } ], "response-transform": { "headers": { "add": [ { "name": "X-custom-response-header", "value": "header-value" } ], "delete": [ "ETag" ] } } } }¶
The ExpressionMatch object contains the rich expression that must evaluate to "True" for the StageMetadata to be applied for the specific StageRules. Defining expressions as stand-alone objects allows for sets of reusable match expressions to be reused via metadata reference linking.¶
Property: expression¶
Example of ExpressionMatch on the referrer and user agent request headers:¶
{ "expression" : "req.h.user-agent *= '*Safari*' and req.h.referrer == 'www.myhost.com'" }¶
The StageMetadata object contains GenericMetadata and HTTP request/response transformations that MUST be applied for a StageRules match. The following table defines the processing stages where request and response transformations are possible:¶
Stage | request-transform | response-transform |
---|---|---|
clientRequest | yes | yes |
originRequest | yes | yes |
originResponse | no | yes |
clientResponse | no | yes |
Note that for the stages where both request and response transformations are allowed, it is possible to specify both. This may be the case if, for example, the request URI needs alteration for cache-key generation and the response headers need to be manipulated.¶
Property: generic-metadata¶
Property: request-transform¶
Property: response-transform¶
Example of a StageMetadata object:¶
{ "generic-metadata" : [{ < Optional list of generic metadata to apply at this stage > }], "request-transform" : { "headers" : { <list of request headers to add/replace/delete> }, "uri" : < URI rewrite, either static or dynamically constructed> } "response-transform" : { "headers" : { <list of response headers to add/replace/delete> }, "response-status" : <Status either static or dynamically constructed > } }¶
The RequestTransform object contains metadata for transforming the HTTP request for a specific StageRules object. The transformation can be the modification of any request header and/or the modification of the URI. Modifications are applied such that downstream processing stages receive the modified HTTP request as their input.¶
Example of a RequestTransform object illustrating a dynamically constructed URI rewrite:¶
{ "request-transform" : { "headers" : { <Optional list of request headers to add/replace/delete> }, "uri" : "req.uri.path", "uri-is-expression" : true }¶
The ResponseTransform object contains metadata for transforming the HTTP response for a StageRules match. The transformation can be the modification of any response header, HTTP response status code, or the generation of a synthetic response. Modifications are applied such that downstream processing stages receive the modified HTTP response as their input.¶
Property: headers¶
Property: response-status¶
Property: status-is-expression¶
Property: synthetic¶
Example of a ResponseTransform object, illustrating a dynamically constructed header value that uses the expression language to concatenate the user agent and host header, and forces a 403 HTTP response status code:¶
{ "response-transform": { "headers": { "add": [ { "name": "X-custom-response-header", "value": "req.h.user-agent . ‘-‘ . req.h.host", "value-is-expressions": true } ] }, "response-status": "403" } }¶
The SyntheticResponse object allows for the specification of a synthetic response to be generated in response to the HTTP request being processed. The synthetic response can contain a set of response headers, a status code, and a response body, and is a complete replacement for any HTTP response elements generated in an earlier processing stage.¶
A dynamically generated Content-Length HTTP response header is generated based on the length of the generated response body.¶
Property: headers¶
Property: response-status¶
Property: status-is-expression¶
Property: body¶
Property: body-is-expression¶
Example of a SyntheticResponse object illustrating a dynamically constructed response body that uses the expression language to combine the request URI with static text and forces a 405 HTTP response status code:¶
{ "headers": [ { "name": "content-type", "value": "text/plain" }, { "name": "X-custom-response-header", "value": "some static value" } ], "response-status": "405", "response-body": "'Sorry, Access to resource '.req.uri.' not allowed'", "body-is-expression": false }¶
The HeaderTransform object specifies how HTTP headers MUST be added, replaced, or deleted from HTTP requests and responses.¶
Property: add¶
Property: replace¶
Property: delete¶
Example of a HeaderTransform object illustrating the addition of two customer headers, the replacement of any previously provided Accept-Encoding header, and the removal of any previously provided Authorization or Accept-Language headers:¶
{ "add": [ { "name": "X-custom-header1", "value": "header-value 1" }, { "name": "X-custom-header2", "value": "header-value 2" } ], "replace": [ { "name": "Accept-Encoding", "value": "gzip,deflate,br" } ], "delete": [ "Authorization", "Accept-Language" ] }¶
The HTTPHeader object contains a name/value pair for an HTTP header to add or replace in a request or response. The CDNI-MEL expression language can be used to dynamically generate response values.¶
Example of an HTTPHeader illustrating a dynamically constructed header value that equals the session parameter from the query string:¶
{ "name": "X-custom-response-header", "value": "req.uri.query.session", "value-is-expression": true }¶
This section documents a set of general purpose GenericMetadata objects whose use and interpretation may be specific to a CDN or Open Caching system's implementation, enabling extensibility and service differentiation for providers.¶
CDN configurations typically have multiple tiers of identifiers that group configurations by customer account to facilitate logging, billing, and support operations. This structure supports two tiers of identifiers (a serviceID which typically identifies a high level customer's service, and a propertyID which typically represents a logical grouping of a set of hosts within a customers' service. It should be noted, however, that the interpretation of ServiceID and PropertyID are implementation-specific, and may not be used by all CDNs and Open Caching systems.¶
This metadata model extension allows for the association service identifier metadata to a host or path match and to allow for these IDs to be dynamically generated via an expression language. For example, it may be necessary to extract a portion of the Request URI path to derive a service identifier (e.g.: /news/* maps to one propertyID and /movies/* maps to a different propertyID). When processing the MI.ServiceIDs metadata for a request, implementations SHOULD override any previously assigned service identifiers with those specified by this metadata.¶
MI.ServiceIDs is a new GenericMetadata object that allows for the specification of the two tiers of CDN-specific service identifiers and service names.¶
Property: service-id¶
Property: service-id-is-expression¶
Property: service-name¶
Property: property-id¶
Property: property-id-is-expression¶
Property: property-name¶
Example illustrating the assignment of a literal service-id along with a dynamically computed property-id that is extracted from the root element of the request URI path.¶
{ "generic-metadata-type": "MI.ServiceIDs", "generic-metadata-value": { "service-id": "12345", "service-name": "My Streaming Service", "property-id": "path_element(req.uri, 1)", "property-id-is-expression": true } }¶
The dCDN may gather a certain number of private features (i.e., not [yet] adopted by SVA or considered marginal) that it may want to expose to the content provider and/or the uCDN. Although private, the announcement, selection, and configuration of this private feature could be done through the OC API.¶
One example could be the support in OCNs of a new protocol that allows the ability to get additional insight about the user agent status (e.g., CTA Wave CMCD).¶
As another example, Broadpeak has developed a feature called S4Streaming, and would like to give the opportunity to control that feature to the uCDN.¶
PrivateFeatureListis a GenericMetadata configuration object as a base generic object that permits the control of private features.¶
Property: features¶
MI.PrivateFeature object contains the following properties:¶
Note that the private features exposed by the dCDN can be advertised through a dedicated FCI object.¶
Example, illustrating the Broadpeak S4 Streaming feature:¶
{ "generic-metadata-type": "MI.PrivateFeatureList", "generic-metadata-value": { "feature": { "feature-oid": "Broadpeak", "feature-type": "S4Streaming", "feature-value": { "footprint": { "footprint-type": "ipv4cidr", "footprint-value": [ "192.0.2.0/24", "198.51.100.0/24" ] }, "activation": "ON", "mode": "transparent", "policy": "bandwidth-max" } } } }¶
The uCDN requires the ability to indicate whether HTTP redirect, DNS redirect, and manifest rewrite are allowed, and indicate which is preferable. This is required in cases where the uCDN would like to delegate the traffic relying on the iterative method but knows the client will not support HTTP redirect. In that case, the uCDN needs a means to force the dCDN to perform request routing based on DNS redirect (or manifest rewrite).¶
This configuration possibility is useful only if the dCDN can advertise the mode of redirection it supports. There is an ongoing discussion in the IETF CDNI group to understand the semantics behind the redirection modes currently in Footprint & Capabilities Advertising Interface (I-DNS and I-HTTP). It is not clear whether this indicates that the dCDN supports one or both delegation modes (the request routing performed by the uCDN can only be based on DNS redirect or HTTP redirect or both), or whether it indicates that the dCDN supports, as its own request routing mode, DNS redirect and/or HTTP redirect. The latter is required for this new configuration object to be valid.¶
MI.RequestRouting is a new GenericMetadata object that allows the uCDN to force the dCDN request routing mode(s) to be applied when working in iterative redirection mode. The list of redirection modes supported by the dCDN is advertised through the FCI.RedirectionMode object. The list of request routing modes supported by the dCDN is advertised through the FCI.RequestRoutingMode object.¶
Property: request-routing-modes¶
Example, illustrating the uCDN forcing the dCDN to use DNS or HTTP as the method for request routing in case the uCDN performs an iterative delegation (i.e., iterative redirection mode):¶
{ "generic-metadata-type": "MI.RequestRouting", "generic-metadata-value": { "request-routing-modes": [ "DNS", "HTTP" ] } }¶
The CDNI Metadata Expression Language provides a syntax with a rich set of variables, operators, and built-in functions to facilitate use cases within the extended CDNi metadata model.¶
Expressions can evaluate to a Boolean, string, or integer, depending on the use case:¶
Usage | Description | Evaluation Results |
---|---|---|
ExpressionMatch.expression | Dynamically determines if StageMetadata should be applied at a specific StageRules. | Boolean. Expressions that do not evaluate to True or False shall be considered as False. |
RequestTransform.uri | Rewrites request URI that will be presented to all downstream stages. | String |
ResponseTransform.response-status | Dynamically sets a response status code to replace the status-code returned by the origin. | Integer (HTTP status code) |
SyntheticResponse.response-status | Dynamically sets a response status code for a synthetically constructed response. | Integer (HTTP status code) |
SyntheticResponse.body | Dynamically constructs a response body. | String |
HTTPHeader.value | Dynamically constructs a header value. | String |
ComputedCacheKey.expression | Dynamically constructs a cache key. | String |
ServiceIDs.properry-id,ServiceIDs.service-id | Dynamically constructs service and property identifiers. | String |
Variable | Meaning |
---|---|
req.h.<name> | Request header <name> |
req.uri | Request URI (includes query string and fragment identifier, if any) |
req.uri.path | Request URI path |
req.uri.pathquery | Request path and query string |
req.uri.query | Request query string |
req.uri.query.<key> | Request query string value associated with <key> |
req.method | Request HTTP method (GET, POST, others) |
resp.h.<name> | Response header <name> |
resp.status | Response status code |
Operator | Type | Result Type | Meaning |
---|---|---|---|
== | infix | Boolean | Equality test |
!= | infix | Boolean | Inequality test |
! | infix | Boolean | Logical NOT operator |
> | infix | Boolean | Greater than test |
< | infix | Boolean | Less than test |
>= | infix | Boolean | Greater than or equal test |
<= | infix | Boolean | Less than or equal |
*= | infix | Boolean | Glob style match |
~= | infix | Boolean | Regular expression match (see https://www.pcre.org/ for details on PCRE RegEx matching) |
ipmatch | infix | Boolean | Match against IP address or CIDR (IPv4 and IPv6) |
+ | infix | Numeric | Addition |
- | infix | Numeric | Subtraction |
* | infix | Numeric | Multiplication |
/ | infix | Numeric | Division |
% | infix | Unsigned or Integer | Modulus |
. | infix | String | Concatenation |
? : | ternary | * | Conditional operator: <e> ? <v1> : <v2> Evaluates <v1> if <e> is true, <v2> otherwise. |
( ) | grouping | Used to override precedence and for function calls. |
Keyword | Meaning |
---|---|
and | Logical AND |
or | Logical OR |
not | Logical NOT (see also the ! operator) |
nil | No value (distinct from empty value) |
true | Boolean constant: true |
false | Boolean constant: false |
Function | Action | Argument(s) | Returns |
---|---|---|---|
integer(e) | Converts expression to integer. | 1 | integer |
real(e) | Converts expression to real. | 1 | real |
string(e) | Converts expression to string. | 1 | string |
boolean(e) | Converts expression to Boolean. | 1 | Boolean |
Function | Action | Argument(s) | Returns |
---|---|---|---|
upper(e) | Converts a string to uppercase. Useful for case-insensitive comparisons. | 1 | string |
lower(e) | Converts a string to lowercase. Useful for case-insensitive comparisons. | 1 | string |
Function | Action | Argument(s) | Returns |
---|---|---|---|
match(string Input, string Match) | Regular expression Match is applied to Input and the matching element (if any) is returned. Empty string is returned if there is no match. See https://www.pcre.org/ for details on PCRE RegEx matching. | 2 | string |
match_replace(string Input, string Match, string Replace) | Regular expression Match is applied to Input arg and replaced with the Replace arg upon successful match. Returns updated (replaced) version of Input. | 3 | string |
add_query(string Input, string q, string v) | Add query string element q with value v to the Input string. If v is nil, then just add the query string element q. The query element q and value v must conform to the format defined in: https://datatracker.ietf.org/doc/html/rfc3986 | 2 | string |
remove_query(string Input, string q) | Remove (all occurrences of) query string element q from the Input string. | 2 | string |
path_element(string Input, integer n) | Return the path element n from Input. -1 returns the last element. | 2 | string |
path_element(string Input, integer n, integer m) | Return the path elements from position n to m. | 3 | string |
To ensure reliable service, all CDNI Metadata configurations MUST be validated for syntax errors before they are ingested into a dCDN. That is, existing configurations should be kept as the live running configuration until the new configuration has passed validation. If errors are detected in a new configuration, the configuration MUST be rejected. A HTTP 500 Internal Server Error should be returned with a message that indicates the source of the error (line number, and configuration element that caused the error).¶
Examples of compile-time errors:¶
If a runtime error is detected when processing a request, the request should be terminated, and a HTTP 500 'Internal Server Error' returned to the caller. To avoid security leaks, sensitive information MUST be removed from the error message before it is returned to an external client. In addition to returning the HTTP 500 error, the dCDN SHOULD log additional diagnostics information to assist in troubleshooting.¶
Examples of runtime errors:¶
Sets the MI.ComputedCacheKey to the value of the X-Cache-Key header from the client request.¶
{ "generic-metadata-type": "MI.ComputedCacheKey", "generic-metadata-value": { "expression": "req.h.x-cache-key" } }¶
Sets the MI.ComputedCacheKey to the lowercase version of the URI.¶
{ "generic-metadata-type": "MI.ComputedCacheKey", "generic-metadata-value": { "expression": "lower(req.uri)" } }¶
ExpressionMatch where the expression is true if the user-agent (glob) matches *Safari* and the referrer equals www.example.com.¶
{ "expression": "req.h.user-agent *= '*Safari*' and req.h.referrer == 'www.example.com'" }¶
Adds X-custom-response-header with a value equal to the value of user-agent - host header.¶
{ "response-transform": { "headers": { "add": [ { "name": "X-custom-response-header", "value": "req.h.user-agent . ' - ' . req.h.host", "value-is-expression": true } ], "response-status": "403" } } }¶
Adds a Set-Cookie header with a dynamically computed cookie value (concatenating user agent and host name) and forces a 403 response.¶
{ "response-transform":{ "headers":{ "add":[ { "name":"Set-Cookie", "value":"req.h.user-agent . ' - ' . req.h.host", "value-is-expression":true } ] } } }¶
Extracts the first path element from the URI. For example, if the URI = /789/second/third/test.txt, property-id is set to the first-path (789).¶
{ "generic-metadata-type":"MI.ServiceIDs", "generic-metadata-value":{ "service-id":"12345", "service-name":"My Streaming Service", "property-id":"path_element(req.uri, 1)", "property-id-is-expression":true } }¶
Since not all dCDNs will be capable of supporting all the extensions proposed in this document, they need the ability to inform uCDNs about their capabilities. [RFC8008] (the CDNI Footprint & Capabilities Interface) was designed for this purpose and is extended here to express these new capabilities.¶
Whenever a capability is represented as a top-level GenericMetadata object, a dCDN will be able to declare its support simply by including that object name in the capability-value list of the standard FCI.Metadata object.¶
For each of the new GenericMetadata objects documented within the SVA Configuration Interface, the default assumption should be that the capability is not supported by the dCDN unless named within the FCI metadata object.¶
Example: A capabilities object declaring support for several of the newly defined GenericMetadata types:¶
{ "capabilities": [ { "capability-type": "FCI.Metadata", "capability-value": { "metadata": [ "MI.SourceMetadataExtended", "MI.ProcessingStages", "MI.CrossoriginPolicy", "MI.CachePolicy", "MI.NegativeCachePolicy", "MI.PrivateFeatureList", "MI.RequestRouting" ] }, "footprints": [ < Footprint Objects > ] } ] }¶
In most cases, the presence or absence of a GenericMetadata object name in FCI.Metadata (as described above), is sufficient to convey support for a capability. There are cases, however, where more fine-grained capabilities declarations are required. Specifically, a dCDN may support some, but not all, of the capabilities specified by one of the new GenericMetadata objects. In these cases, new FCI objects will be created to allow a dCDN to express these fine-grained capabilities.¶
The AuthTypes object is used to indicate the support of authentication methods to be used for content acquisition (while interacting with an origin server) and authorization methods to be used for content delivery.¶
This specification document defines two new authentication methods (see MI.Auth) while there is one authorization method currently under specification in CDNI called [URI.signing]¶
FCI.AuthTypes example:¶
{ "capabilities": [ { "capability-type": "FCI.AuthTypes", "capability-value": { "authe-types": [ "AWSv4Auth", "HeaderAuth" ], "autho-types": [ "UriSigning" ] } } ] }¶
This object is used to signal the set of features that are supported in relation to the ProcessingStages configuration object (see MI.ProcessingStages). Those optional features depend on the CDNI-MEL language support.¶
Property: features¶
Example:¶
{ "capabilities": [ { "capability-type": "FCI.ProcessingStages", "capability-value": { "features": [ "ExpressionMatch", "RequestTransform", "ResponseTransform" ] } } ] }¶
This object is used to signal the supported features related to the SourceMetadataExtended configuration object.¶
Property: load-balance¶
If the FCI.SourceMetadtaExtended object is not exposed/advertised or if the "load-balance" array is empty, the dCDN does not support the usage of the load-balance property attached to the SourceMetadataExtended configuration object (see MI.SourceMetadataExtended).¶
Example:¶
{ "capabilities": [ { "capability-type": "FCI.SourceMetadataExtended", "capability-value": { "load-balance": [ "random", "content-hash", "ip-hash" ] } } ] }¶
This object is used by the dCDN to signal/announce the supported request routing modes. This can be optionally used by the uCDN to further select a subset of those modes when operating one of the iterative delegation modes. See the section on the GenericMetadata RequestRouting object..¶
Property: request-routing-modes¶
Example:¶
{ "capabilities": [ { "capability-type": "FCI.RequestRouting", "capability-value": { "request-routing-modes": [ "DNS", "HTTP", "MANIFEST_REWRITE" ] } } ] }¶
This object is used by the dCDN to signal/announce the list of supported private features. See the section on the GenericMetadata PrivateFeatureList object.¶
Property: features¶
Example:¶
{ "capabilities": [ { "capability-type": "FCI.PrivateFeatures", "capability-value": { "features": [ { "feature-oid": "Broadpeak", "feature-type": "S4Streaming" } ] } } ] }¶
This object contains the following properties:¶
This object is used by the dCDN to signal/announce the supported OCN types and/or their transport arrangement and/or medium supported by OCNs.¶
Property ocn-delivery-list¶
o Description: List of supported OCN types. Examples include: "HOME" or "EDGE".¶
o Type: Array of strings¶
o Mandatory-to-Specify: No¶
This document requests the registration of the following entries under the "CDNI Payload Types" registry hosted by IANA¶
Payload type | Specification |
---|---|
MI.CachePolicy | RFCthis |
MI.NegativeCachePolicy | RFCthis |
MI.StaleContentCachePolicy | RFCthis |
MI.CacheBypassPolicy | RFCthis |
MI.ComputedCacheKey | RFCthis |
MI.AllowCompress | RFCthis |
MI.SourceMetadataExtended | RFCthis |
MI.SourceExtended | RFCthis |
MI.LoadBalanceMetadata | RFCthis |
MI.HeaderAuth | RFCthis |
MI.AWSv4Auth | RFCthis |
MI.CrossOriginPolicy | RFCthis |
MI.AuthTokenMetadata (TBD) | RFCthis |
MI.CertificateMetadata (TBD) | RFCthis |
MI.OcnSelection | RFCthis |
MI.RequestRouting | RFCthis |
MI.ProcessingStages | RFCthis |
MI.StageRules | RFCthis |
MI.ExpressionMatch | RFCthis |
MI.StageMetadata | RFCthis |
MI.RequestTransform | RFCthis |
MI.ResponseTransform | RFCthis |
MI.SyntheticResponse | RFCthis |
MI.HeaderTransform | RFCthis |
MI.HTTPHeader | RFCthis |
MI.ServiceIDs | RFCthis |
MI.TrafficType | RFCthis |
MI.LoggingMetadata (TBD) | RFCthis |
MI.PrivateFeatureList | RFCthis |
FCI.AuthTypes | RFCthis |
FCI.ProcessingStages | RFCthis |
FCI.SourceMetadataExtended | RFCthis |
FCI.RequestRouting | RFCthis |
FCI.PrivateFeatures | RFCthis |
FCI.OcnSelection | RFCthis |
This specification is in accordance with the CDNI Request Routing: Footprint and Capabilities Semantics. As such, it is subject to the security and privacy considerations as defined in Section 8 of [RFC8006] and in Section 7 of [RFC8008] respectively.¶
This document presents requirements and extensions to the CDNI metadata model to cover typical use cases found in the commercial CDN and Open Caching ecosystems. By limiting the scope of these extensions to new GenericMetadata objects, backward compatibility can be maintained with any existing CDNI Metadata Interface implementations.¶