We provide a way for all Akamai customers to stream logs to our platform, as well as configure agent site.

Allow TollBit's IP Addresses

TollBit sends these requests from a fixed set of static IP addresses, published at https://tollbit.com/static-ips.txt. At the time of writing, the list is:

52.22.183.94
3.220.109.109

Always use the published list as the source of truth, and check it again if TollBit tells you it has changed.

Our requests use the user agent tollbot/1.0, so you may see it show up in your logs.

Add these addresses to an IP network list in Akamai. Then make sure that list is allowed through every security layer that could block or challenge our requests. This applies to every hostname you onboard with us:

• App & API Protector / Kona Site Defender: IP/Geo Firewall, WAF rule exceptions, and rate controls.
• Bot Manager, if enabled: make sure requests from these IPs are not denied, challenged, tarpitted, or served alternate content.
• Any firewall or WAF in front of your origin servers, outside of Akamai.

If you are unsure whether your configuration blocks us, contact [email protected] and we will confirm from our side.

Steps for Analytics

Create a Stream with DataStream 2

You will need to first create a stream by going to your Akamai Control Center. Follow these instructions on how to create your stream.

Choose Data Parameters

When choosing data parameters, make sure to choose parameters that cover at least everything in the following sample log JSON. Also, please ensure that your log format is JSON.

{
  "version": 1,
  "streamId": "12345",
  "cp": "123456",
  "reqId": "1239f220",
  "reqTimeSec": "1573840000",
  "bytes": "4995",
  "cliIP": "128.147.28.68",
  "statusCode": "206",
  "proto": "HTTPS",
  "reqHost": "test.hostname.net",
  "reqMethod": "GET",
  "reqPath": "/path1/path2/file.ext",
  "reqPort": "443",
  "rspContentLen": "5000",
  "rspContentType": "text/html",
  "UA": "Mozilla%2F5.0+%28Macintosh%3B+Intel+Mac+OS+X+10_14_3%29",
  "tlsOverheadTimeMSec": "0",
  "tlsVersion": "TLSv1",
  "objSize": "484",
  "uncompressedSize": "484",
  "overheadBytes": "232",
  "totalBytes": "0",
  "queryStr": "param=value",
  "breadcrumbs": "//BC/%5Ba=23.33.41.20,c=g,k=0,l=1%5D",
  "accLang": "en-US",
  "range": "37334-42356",
  "referer": "https%3A%2F%2Ftest.referrer.net%2Fen-US%2Fdocs%2FWeb%2Ftest",
  "xForwardedFor": "8.47.28.38",
  "maxAgeSec": "3600",
  "reqEndTimeMSec": "3",
  "errorCode": "ERR_ACCESS_DENIED|fwd_acl",
  "turnAroundTimeMSec": "11",
  "transferTimeMSec": "125",
  "dnsLookupTimeMSec": "50",
  "lastByte": "1",
  "country": "IN",
  "state": "Virginia",
  "city": "HERNDON"
}

Stream to Endpoint

To forward your logs to us, follow the steps outlined here. The endpoint url that you should be streaming to is https://log.tollbit.com/log/akamai.

Select none for authentication for now, as we will be setting up custom authentication. To do so, go to "Custom header". For the content type, you can select application/json. Add a new header value with the key TollbitKey and the value as your secret key from your dashboard.

Finally, you can review and activate your stream!

Steps for Agent Site

Akamai allows you to set up redirection rules at the edge using either Cloudlets or Content Protector.

• For Cloudlets, please see the steps outlined below.
• For Content Protector, please reach out to your Akamai account team and loop in [email protected] as well.

Cloudlets Set Up

To set up agent sites for Akamai with Cloudlets, you would use their Forward Rewrite Cloudlet. Follow the documentation here for reference.

Setup New Origin
You’ll need to set up a new conditional origin that is your tollbit subdomain to have the forward rewrite cloudlet correctly route the request. Follow the docs here on doing this.

At a high level, you would go to Property Manager and create a new Conditional Origin with the server domain name url as your tollbit subdomain (i.e. tollbit.example.com). In the origin's settings, set Forward Host Header to Origin Hostname — TollBit serves agent content based on the tollbit subdomain hostname, so rewritten requests should arrive with the Host header set to tollbit.example.com rather than your site's incoming hostname. Ensure that this origin is activated and deployed.

Forward Rewrite Cloudlet
Create a new forward rewrite policy by following the docs here. Then create a forward rewrite rule following the docs here. For the match type, you want to match on if the request header’s User-Agent header contains one of the following user agents (case insensitive):

'AgentNativeSearchBot', 'AI2Bot', 'AionBot', 'AIWebIndex', 'Amazonbot', 'Amzn-SearchBot', 'Amzn-User', 'anthropic-ai', 'AteveSearchSourceUrlDiscovery', 'AzureAI-SearchBot', 'Bravebot', 'Bytespider', 'CCBot', 'ChatGPT-User', 'claude-code', 'Claude-SearchBot', 'Claude-User', 'Claude-Web', 'Claude/', 'ClaudeBot', 'CloudflareBrowserRenderingCrawler', 'cohere-ai', 'Comet/', 'Cursor/', 'DesearchBot', 'Devin/', 'Diffbot', 'DuckAssistBot', 'Exabot', 'ExaSearchBot', 'ExaUser', 'Fellou', 'Gensparkbot', 'GitHubCopilotRuntime', 'goose/', 'GPTBot', 'grok-agent', 'GrokBot', 'Hermes-Agent', 'iaskspider', 'ibou-search', 'IbouBot', 'IsonSearchBot', 'Keenable', 'Kimi-SearchBot', 'KimiBot', 'Lightpanda', 'LinkupBot', 'Manus-User', 'meta-externalagent', 'meta-externalfetcher', 'meta-webindexer', 'MistralAI-User', 'MojeekBot', 'MySearchBot', 'OAI-AdsBot', 'OAI-SearchBot', 'OpenClaw', 'Perplexity-User', 'Perplexity/', 'PerplexityBot', 'rednote-websearch-bot', 'Reflectionbot', 'Shap-User', 'ShapBot', 'SofyaBot', 'TalarionSearchBot', 'Timpibot', 'YouBot'

Ensure that the rewrite points to the new tollbit subdomain origin you added, while preserving the original URL path and query strings.

To verify the setup, request a page on your site with one of the user agents above (for example curl -A "GPTBot" https://example.com/some-article) and confirm the response body contains the agent version of the page. Once you’ve tested this appropriately, you can activate and deploy.