HTTP#
Making requests#
Hint
The finished example can be found here
Note
We will use await at the top level here as its easier to explain. For your own code, please use a async function.
If you want a example of how it can be done in a async function, see the full example.
Setting up#
from os import environ
from discord_typings import ChannelData
from nextcore.http import BotAuthentication, HTTPClient, Route
# Constants
AUTHENTICATION = BotAuthentication(environ["TOKEN"])
CHANNEL_ID = environ["CHANNEL_ID"]
# HTTP Client
http_client = HTTPClient()
await http_client.setup()
Note
You will need to set environment variables on your system for this to work.
export TOKEN="..."
export CHANNEL_ID="..."
$env:TOKEN = "..."
$env:CHANNEL_ID = "..."
Creating a Route
#
First you need to find what route you are going to implement. A list can be found on https://discord.dev
For this example we are going to use Get Channel
For the first parameter, this will be the HTTP method.
route = Route("GET", ...)
The second parameter is the “route”. This is the path of the request, without any parameters included.
To get this, you take the path (/channels/{channel.id}
) and replace .
with _
It will look something like this.
route = Route("GET", "/channels/{channel_id}", ...)
Warning
This should not be a f-string!
The kwargs will be parameters to the path.
route = Route("GET", "/channels/{channel_id}", channel_id=CHANNEL_ID)
Doing the request#
To do a request, you will need to use HTTPClient.request()
.
response = await http_client.request(route,
rate_limit_key=AUTHENTICATION.rate_limit_key,
headers=AUTHENTICATION.headers
)
This will return a aiohttp.ClientResponse
for you to process.
Getting the response data#
We can use aiohttp.ClientResponse.json()
to get the JSON response data.
channel = await response.json()
And finally, get the channel name
print(channel.get("name")) # DM channels do not have names
Cleaning up#
When you are finished with all requests, you can close the HTTP client gracefully.
await http_client.close()
HTTP reference#
- class nextcore.http.HTTPClient(*, trust_local_time=True, timeout=60, max_rate_limit_retries=10)#
The HTTP client to interface with the Discord API.
Example usage
http_client = HTTPClient() await http_client.setup() # This can be found on https://discord.dev/topics/gateway#get-gateway route = Route("GET", "/gateway") # No authentication is used, so rate_limit_key None here is equivilent of your IP. response = await http_client.request(route, rate_limit_key=None) gateway: GetGatewayData = await response.json() print(gateway["url"]) await http_client.close()
- Parameters:
trust_local_time (bool) – Whether to trust local time. If this is not set HTTP rate limiting will be a bit slower but may be a bit more accurate on systems where the local time is off.
timeout (float) – The default request timeout in seconds.
max_rate_limit_retries (int) – How many times to attempt to retry a request after rate limiting failed.
- trust_local_time#
If this is enabled, the rate limiter will use the local time instead of the discord provided time. This may improve your bot’s speed slightly.
Warning
If your time is not correct, and this is set to
True
, this may result in more rate limits being hit.You can check if your clock is synchronized by running the following command:
timedatectl
If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”
If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.
To enable the ntp service run the following command:
sudo timedatectl set-ntp on
This will automatically sync the system clock every once in a while.
You can check if your clock is synchronized by running the following command:
timedatectl
If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”
If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.
To enable the ntp service run the following command:
sudo timedatectl set-ntp on
This will automatically sync the system clock every once in a while.
This can be turned on by going to
Settings -> Time & language -> Date & time
and turning onSet time automatically
.
- timeout#
The default request timeout in seconds.
- default_headers#
The default headers to pass to every request.
- max_retries#
How many times to attempt to retry a request after rate limiting failed.
Note
This does not retry server errors.
- rate_limit_storages#
Classes to store rate limit information.
The key here is the rate_limit_key (often a user ID).
- async close()#
Clean up internal state
- Return type:
None
- async connect_to_gateway(*, version=UndefinedType.UNDEFINED, encoding=UndefinedType.UNDEFINED, compress=UndefinedType.UNDEFINED)#
Connects to the gateway
Example usage:
ws = await http_client.connect_to_gateway()
- Parameters:
version (Literal[6, 7, 8, 9, 10] | UndefinedType) –
The major API version to use
Hint
It is a good idea to pin this to make sure something doesn’t unexpectedly change
encoding (Literal['json', 'etf'] | UndefinedType) – Whether to use json or etf for payloads
compress (Literal['zlib-stream'] | UndefinedType) – Payload compression from data sent from Discord.
- Raises:
RuntimeError –
HTTPClient.setup()
was not called yet.RuntimeError – HTTPClient was closed.
- Returns:
The gateway websocket
- Return type:
- async request(route, rate_limit_key, *, headers=None, bucket_priority=0, global_priority=0, wait=True, **kwargs)#
Requests a route from the Discord API
- Parameters:
route (Route) – The route to request
rate_limit_key (str | None) –
A ID used for differentiating rate limits. This should be a bot or oauth2 token.
Note
This should be
None
for unauthenticated routes or webhooks (does not include modifying the webhook via a bot).headers (dict[str, str] | None) – Headers to mix with
HTTPClient.default_headers
to pass toaiohttp.ClientSession.request()
bucket_priority (int) – The request priority to pass to
Bucket
. Lower priority will be picked first.global_priority (int) –
The request priority for global requests. Lower priority will be picked first.
Warning
This may be ignored by your
BaseGlobalRateLimiter
.wait (bool) –
Wait when rate limited.
This will raise
RateLimitedError
if set toFalse
and you are rate limited.kwargs (Any) – Keyword arguments to pass to
aiohttp.ClientSession.request()
- Returns:
The response from the request.
- Return type:
ClientResponse
- Raises:
RuntimeError –
HTTPClient.setup()
was not called yet.RuntimeError – HTTPClient was closed.
RateLimitedError – You are rate limited, and
wait
was set toFalse
CloudflareBanError – You have been temporarily banned from the Discord API for 1 hour due to too many requests. Read the documentation for more information.
BadRequestError – The request data was invalid.
UnauthorizedError – No token was provided on a endpoint that requires a token.
ForbiddenError – The token was valid but you do not have permission to use do this.
NotFoundError – The endpoint you requested was not found or a route parameter was invalid.
InternalServerError – Discord is having issues. Try again later.
HTTPRequestStatusError – A non-200 status code was returned.
- async setup()#
Sets up the HTTP session
Warning
This has to be called before
HTTPClient._request()
orHTTPClient.connect_to_gateway()
- Raises:
RuntimeError – This can only be called once
- Return type:
None
- class nextcore.http.Route(method, path, *, ignore_global=False, guild_id=None, channel_id=None, webhook_id=None, webhook_token=None, **parameters)#
Metadata about a discord API route
Example usage
route = Route("GET", "/guilds/{guild_id}", guild_id=1234567890)
- Parameters:
method (Literal['GET', 'HEAD', 'POST', 'PUT', 'DELETE', 'CONNECT', 'OPTIONS', 'TRACE', 'PATCH']) – The HTTP method of the route
path (LiteralString) – The path of the route. This can include python formatting strings ({var_here}) from kwargs
ignore_global (bool) – If this route bypasses the global rate limit.
guild_id (Snowflake | None) – Major parameters which will be included in
parameters
and count towards the rate limit.channel_id (Snowflake | None) – Major parameters which will be included in
parameters
and count towards the rate limit.webhook_id (Snowflake | None) – Major parameters which will be included in
parameters
and count towards the rate limit.webhook_token (str | None) – Major parameters which will be included in
parameters
and count towards the rate limit.parameters (Snowflake) –
The parameters of the route. These will be used to format the path.
This will be included in
Route.bucket
- method#
The HTTP method of the route
- route#
The path of the route. This can include python formatting strings ({var_here}) from kwargs.
- path#
The formatted version of
Route.route
- ignore_global#
If this route bypasses the global rate limit.
This is always
True
for unauthenticated routes.
- bucket#
The rate limit bucket this fits in.
This is created from
Route.guild_id
,Route.channel_id
,Route.webhook_id
,Bucket.method
andRoute.path
- class nextcore.http.RateLimitStorage#
Storage for rate limits for a user.
One of these should be created for each user.
Note
This will register a gc callback to clean up the buckets.
- global_lock#
The users per user global rate limit.
- async close()#
Clean up before deletion.
Warning
If this is not called before you delete this or it goes out of scope, you will get a memory leak.
- Return type:
None
- async get_bucket_by_discord_id(discord_id)#
Get a rate limit bucket from the Discord bucket hash.
This can be obtained via the
X-Ratelimit-Bucket
header.
- async get_bucket_by_nextcore_id(nextcore_id)#
Get a rate limit bucket from a nextcore created id.
- Parameters:
nextcore_id (str) – The nextcore generated bucket id. This can be gotten by using
Route.bucket
- Return type:
Bucket | None
- async get_bucket_metadata(bucket_route)#
Get the metadata for a bucket from the route.
- Parameters:
bucket_route (str) – The bucket route.
- Return type:
BucketMetadata | None
- async store_bucket_by_discord_id(discord_id, bucket)#
Store a rate limit bucket by the discord bucket hash.
This can be obtained via the
X-Ratelimit-Bucket
header.
- async store_bucket_by_nextcore_id(nextcore_id, bucket)#
Store a rate limit bucket by nextcore generated id.
- async store_metadata(bucket_route, metadata)#
Store the metadata for a bucket from the route.
- Parameters:
bucket_route (str) – The bucket route.
metadata (BucketMetadata) – The metadata to store.
- Return type:
None
Authentication#
- class nextcore.http.BaseAuthentication#
A wrapper around discord credentials.
Warning
This is a base class. You should probably use
BotAuthentication
orBearerAuthentication
instead.Example implementation
class BotAuthentication(BaseAuthentication): """A wrapper around bot token authentication. **Example usage** .. code-block:: python3 authentication = BotAuthentication(os.environ["TOKEN"]) route = Route("GET", "/gateway/bot") await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers) Parameters ---------- token: The bot token. Attributes ---------- prefix: The prefix of the token. token: The bot token """ __slots__: tuple[str, ...] = () def __init__(self, token: str) -> None: self.prefix: Literal["Bot"] = "Bot" self.token: str = token @property def rate_limit_key(self) -> str: """The key used for rate limiting This will be in the format ``Bot AABBCC.DDEEFF.GGHHII`` **Example usage** .. code-block:: python3 await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...) """ return f"{self.prefix} {self.token}" @property def headers(self) -> dict[str, str]: """Headers for doing a authenticated request. This will return a dict with a ``Authorization`` field. **Example usage** .. code-block:: python3 await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...) """ return {"Authorization": f"{self.prefix} {self.token}"}
- prefix#
The prefix of the authentication.
- token#
The bot’s token.
- abstract property headers: dict[str, str]#
Headers used for making a authenticated request.
This may return a empty dict if headers is not used for authenticating this type of authentication.
Example usage
await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)
- class nextcore.http.BotAuthentication(token)#
A wrapper around bot token authentication.
Example usage
authentication = BotAuthentication(os.environ["TOKEN"]) route = Route("GET", "/gateway/bot") await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers)
- Parameters:
token (str) – The bot token.
- prefix#
The prefix of the token.
- token#
The bot token
- class nextcore.http.BearerAuthentication(token)#
A wrapper around OAuth2 Bearer Token authentication.
- Parameters:
token (str) – The bearer token.
- prefix#
The prefix of the token.
- token#
The bearer token
Bucket rate limiting#
- class nextcore.http.Bucket(metadata)#
A discord rate limit implementation around a bucket.
Example usage
bucket_metadata = BucketMetadata() bucket = Bucket(bucket_metadata) async with bucket.acquire(): # Do request await bucket.update(remaining, reset_after_seconds, unlimited=False)
- Parameters:
metadata (BucketMetadata) – The metadata for the bucket.
- metadata#
The metadata for the bucket.
This should also be updated with info that applies to all buckets like limit and if it is unlimited.
- reset_offset_seconds#
How much the resetting should be offset to account for processing/networking delays.
This will be added to the reset time, so for example a offset of
1
will make resetting 1 second slower.
- acquire(*, priority=0, wait=True)#
Use a spot in the rate limit.
Example usage
async with bucket.acquire(): # Do request await bucket.update(remaining, reset_after_seconds, unlimited=False)
- Parameters:
- Raises:
RateLimitedError – You are rate limited and
wait
was set toFalse
- Return type:
AsyncIterator[None]
- async close()#
Cleanup this instance.
This should be done when this instance is never going to be used anymore
Warning
Continued use of this instance will result in instability
- property dirty: bool#
Whether the bucket is currently any different from a clean bucket created from a
BucketMetadata
.This can be for example if any requests is being made, or if the bucket is waiting for a reset.
- class nextcore.http.BucketMetadata(limit=None, *, unlimited=False)#
Metadata about a discord bucket.
Example usage
bucket_metadata = BucketMetadata() bucket = Bucket(bucket_metadata) async with bucket.acquire(): ... bucket_metadata.limit = 5 # This can be found in the response headers from discord. bucket_metadata.unlimited = False
- Parameters:
- limit#
The maximum number of requests that can be made in the given time period.
Note
This will be
None
ifBucketMetadata.unlimited
isTrue
.This will also be
None
if no limit has been fetched yet.
- unlimited#
Wheter the bucket has no rate limiting enabled.
- class nextcore.http.RequestSession(*, priority=0, unlimited=False)#
A metadata class about a pending request. This is used by
Bucket
- Parameters:
- pending_future#
The future that when set will execute the request.
- priority#
The priority of the request. Lower is better!
- unlimited#
If this request was made when the bucket was unlimited.
This exists to make sure that there is no bad state when switching between unlimited and limited.
Global rate limiting#
- class nextcore.http.BaseGlobalRateLimiter#
A base implementation of a rate-limiter for global-scoped rate-limits.
Warning
This does not contain any implementation!
You are probably looking for
LimitedGlobalRateLimiter
orUnlimitedGlobalRateLimiter
- abstract acquire(*, priority=0, wait=True)#
Use a spot in the rate-limit.
- Parameters:
- Returns:
A context manager that will wait in __aenter__ until a request should be made.
- Return type:
- abstract async close()#
Cleanup this instance.
This should be done when this instance is never going to be used anymore
Warning
Continued use of this instance may result in instability
- Return type:
None
- class nextcore.http.LimitedGlobalRateLimiter(limit=50)#
A limited global rate-limiter.
- Parameters:
limit (int) – The amount of requests that can be made per second.
- class nextcore.http.UnlimitedGlobalRateLimiter#
A global rate-limiting implementation
This works by allowing infinite requests until one fail, and when one fails stop further requests from being made until
retry_after
is done.Warning
This may cause a lot of 429’s. Please use
LimitedGlobalRateLimiter
unless you are sure you need this.Warning
This is slower than other implementations due to using Discord’s time.
There is some extra delay due to ping due to this.
- acquire(*, priority=0, wait=True)#
Acquire a spot in the rate-limit
- Parameters:
- Raises:
RateLimitedError –
wait
was set toFalse
and we are rate limited.- Returns:
A context manager that will wait in __aenter__ until a request should be made.
- Return type:
- async close()#
Cleanup this instance.
This should be done when this instance is never going to be used anymore
Warning
Continued use of this instance will result in instability
- Return type:
None
HTTP errors#
- exception nextcore.http.RateLimitingFailedError(max_retries, response)#
When rate limiting has failed more than
HTTPClient.max_retries
timesHint
This can be due to a un-syncronized clock.
You can change
HTTPClient.trust_local_time
toFalse
to disable using your local clock, or you could sync your clock.You can check if your clock is synchronized by running the following command:
timedatectl
If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”
If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.
To enable the ntp service run the following command:
sudo timedatectl set-ntp on
This will automatically sync the system clock every once in a while.
You can check if your clock is synchronized by running the following command:
timedatectl
If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”
If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.
To enable the ntp service run the following command:
sudo timedatectl set-ntp on
This will automatically sync the system clock every once in a while.
This can be turned on by going to
Settings -> Time & language -> Date & time
and turning onSet time automatically
.- Parameters:
max_retries (int) – How many retries the request used that failed.
response (ClientResponse) – The response to the last request that failed.
- Return type:
None
- max_retries#
How many retries the request used that failed.
- response#
The response to the last request that failed.
- exception nextcore.http.CloudflareBanError#
A error for when you get banned by cloudflare
This happens due to getting too many
401
,403
or429
responses from discord. This will block your access to the API temporarily for an hour.See the documentation for more info.
- exception nextcore.http.HTTPRequestStatusError(error, response)#
A base error for receiving a status code the library doesn’t expect.
- Parameters:
error (HTTPErrorResponseData) – The error json from the body.
response (ClientResponse) – The response to the request.
- Return type:
None
- response#
The response to the request.
- error_code#
The error code.
- message#
The error message.
- error#
The error json from the body.
- exception nextcore.http.BadRequestError(error, response)#
A 400 error.
- Parameters:
error (HTTPErrorResponseData) –
response (ClientResponse) –
- Return type:
None
- exception nextcore.http.NotFoundError(error, response)#
A 404 error.
- Parameters:
error (HTTPErrorResponseData) –
response (ClientResponse) –
- Return type:
None
- exception nextcore.http.UnauthorizedError(error, response)#
A 401 error.
- Parameters:
error (HTTPErrorResponseData) –
response (ClientResponse) –
- Return type:
None
- exception nextcore.http.ForbiddenError(error, response)#
A 403 error.
- Parameters:
error (HTTPErrorResponseData) –
response (ClientResponse) –
- Return type:
None
- exception nextcore.http.InternalServerError(error, response)#
A 5xx error.
- Parameters:
error (HTTPErrorResponseData) –
response (ClientResponse) –
- Return type:
None