Skip to main content

Optimizing Large Audio File Delivery

Overview​

This use case demonstrates how to optimize the delivery of large audio files stored in Cloudflare R2 using Cloudflare Workers and Web Application Firewall (WAF) rules to ensure secure and efficient access. This approach leverages Cloudflare's caching capabilities without relying on the Cache API.

Architecture​

Components​

  • Cloudflare CDN: Serves audio files to platforms such as Apple Music and Spotify.
  • Cloudflare Workers: Executes custom logic to manage access and security with HMAC token.
  • R2 Storage: Cloudflare's object storage solution.
  • Web Application Firewall (WAF): Protects the R2 bucket by validating HMAC tokens.

Problem Statement​

Initially, using Cloudflare R2 and Cloudflare Workers together posed a significant challenge due to the limitations of the S3 endpoint, which is not compatible with a public hostname and consequently, with Cloudflare's caching capabilities. To achieve private access, the Cache API had to be used. However, the primary limitation was not with the Cache API itself but with Cloudflare Workers, which have a memory limit of 128MB. When attempting to cache large audio files using the Cache API, the entire file needed to be loaded into the worker's memory, often exceeding the available memory and leading to performance issues and failed requests.

An alternative solution could be to use the Cloudflare Stream API, which handles video and audio streaming efficiently. However, for large audio files, this approach does not support range requests when reading the cached conetent, which are essential for optimal performance in many applications. Without the ability to perform range requests, the app's performance decreases significantly, as users cannot quickly access specific parts of the audio files. This makes the Stream API less suitable for use cases requiring efficient handling of large audio files with partial content requests.

Solution​

The solution involves associating a public hostname with the R2 bucket and protecting this hostname with WAF rules that validate HMAC tokens. This setup allows the use of Cloudflare's caching capabilities without directly invoking the Cache API, ensuring optimal performance and security.

Implementation Steps​

1. Link Public Hostname to R2 Bucket:​

  • Configure a public hostname to point to your R2 bucket.

2. Create WAF Rule for HMAC Validation:​

  • Set up a WAF rule to block any requests that do not have a valid HMAC token.

3. Generate HMAC Token:​

4. Test Configuration:​

Test the setup to ensure that requests without a valid token are blocked, and requests with a valid token are allowed.

  • Invalid Token:

curl -X GET "https://your-public-hostname/path-to-file?verify=invalid_token"

Expected result: 403 Forbidden

  • Valid Token:

curl -X GET "https://your-public-hostname/path-to-file?verify=valid_token"

Expected result: 200 OK with the requested file

5. Performance Testing:​

Perform tests to ensure that the audio file delivery is efficient and that partial content requests are handled swiftly:

curl -X GET "https://your-public-hostname/path-to-file?verify=valid_token" -H "Range: bytes=0-1048576"

Expected result: 206 Partial Content with a quick response time

6. (Optional) Redirect Rules for Path Management:​

  • Use Redirect Rules to simplify the management of multiple buckets by redirecting paths to specific hostnames. This allows you to expose a single domain to customers/providers and manage the internal mappings for multiple buckets:

https://your-public-hostname/eu-west -> https://eu-west.your-public-hostname

Conclusion​

By associating a public hostname with your R2 bucket and protecting it with WAF rules for HMAC validation, you can leverage Cloudflare's caching capabilities effectively. Additionally, using Redirect Rules helps in managing multiple buckets seamlessly. This approach ensures secure and efficient delivery of large audio files, optimizing performance for partial content requests and reducing load on the network.

For further details, refer to: