Content
Content represents the actual payload and data that you are looking for as a developer. You get content by making a request to a specific webpage. Every webpage has its own rate.
Response
- Name
content- Type
- object
- Description
Object containing the different sections of the page.
- Name
metadata- Type
- string
- Description
Additional information not within the page that may provide additional contextual information. May include follow up links or topics to suggest to the end user to explore.
Content
- Name
header- Type
- string
- Description
The header of the page. This contains information such as navigation links and other auxillary information not strictly related to the main purpose of the page.
- Name
main- Type
- object
- Description
The main content of the page, excluding any headers, navigational components, or footers, rendered in markdown for easy parsing.
- Name
footer- Type
- string
- Description
The footer of the page. This contains information such as follow up articles, terms of service, and social media links.
Get Content
This endoint allows you to manually request content of a certain webpage. The <content_path> is simply the full url
of the intended webpage.
Required attributes
- Name
orgCuid- Type
- string
- Description
Account identifier that can be found in the access page of the dashboard.
- Name
token- Type
- string
- Description
The generated token used to access data one time.
- Name
userAgent- Type
- string
- Description
Unique AgentID string for who is allowed to access the page. Make sure to use the AgentID that you will use within your application.
Request
import Tollbit from 'tollbit-sdk'
const client = new Tollbit(SECRET_KEY, ORGANIZATION_ID, USER_AGENT)
const response = await client.getContent({
url: "https://joshmayer.net/posts/bix",
maxPriceMicros: 11000000,
currency: "USD",
licenseType: "ON_DEMAND_LICENSE",
})
Response
{
"content": {
"header": "...",
"main": "December 10, 2023
Up until now, the internet has successfully differentiated humans and bots (at least enough to rely on). But in the age of AI this will not work. Captchas are easily bypassed, and eventually
any user interaction on the internet will probably be navigable by an AI agent. Given this future, why continue to distinguish the two? What were to happen if you opened up the
flood gates to bot mania? Imo, it might actually work to build a newer, more efficient layer of the internet where you dont have to actually navigate page to page. Instead all
a human needs to do is ask for exactly what they want.
When you think about what you do daily on the internet today you probalu think about scrolling on TikTok, X, or checking your email.
Obviously, there are many concerns with such a drastic change to the internet.",
"footer": "..."
},
"metadata": "...",
"rate": {
"priceMicros": 20000,
"currency": "USD",
"licenseType": "ON_DEMAND_LICENSE",
"licensePath": "...",
"error": ""
}
}