Too Many Requests? Understand the Art of Rate Limit

One fine day, we decided to develop a face recognition system at my company, and I was eager to dive into the coding process.

Little did I know that I would soon find myself trapped in the dreaded realm of 'Too Many Requests.' The reason? I was making numerous calls to third-party APIs, each with its distinct rate limit per second.

However, I persevered, and eventually, I managed to fine-tune the entire system to operate as smoothly as a car cruising along an empty road.

To overcome this challenge, I implemented rate limiting using Google Cloud Tasks.

WHAT IS GOOGLE CLOUD TASKS?

Think of it as a supervisor that monitors all the APIs you're using. It ensures that your system follows certain rules, such as how many APIs can be used simultaneously and how often they can be called within a specific time frame.

From a coding perspective, It's similar to sending an email without having to worry about anything at all.

Lets dig into the Google Cloud Platform and take a look at it.

This is how the Cloud Tasks user interface looks as of the day I'm writing this blog.

Let's click on 'Create Push Queue' and explore its functionality.

Let's break down the meaning of all these technical terms one by one.

Max Dispatches: This is the maximum number of API calls we want to make per second.

Note: If you want to make, for example, 20 calls per minute, which translates to 0.333 calls per second, configuring it directly from the GCP console is not possible.

Instead, you should use the Google Cloud Tasks SDK or the Google CLI for this specific configuration.

Max Concurrent Dispatches: This is the maximum number of simultaneous API calls that can be made.

Retry Configuration: it is used to determine how many times a task should retry in case the API encounters an error.

I won't dive into all the details here for Retry Configuration; consider this a personal assignment to explore and learn more on your own. 😉

Now, let's go ahead and create a queue and write some actual code.

I'll be using Node.js to add tasks to the queue.

install the module and import it

npm i @google-cloud/tasks

const { CloudTasksClient } = require("@google-cloud/tasks");

const main = () => {
     const payload = {"data":"PUT YOUR ACTUAL JSON HERE"}
    const request = {
        parent: `projects/${projectId}/locations/us-west2/queues/my-rate-limit-saviour`,
        task: {
          httpRequest: {
            httpMethod: "POST",
            url: `API TO CALL THIS CAN BE YOUR BACKEND API OR CLOUD FUNCTIONS`,
            body: Buffer.from(JSON.stringify(payload)).toString("base64"),
            headers: { "Content-Type": "application/json" },
          },
        },
     };
      try {
            const client = new CloudTasksClient();
            const [response] = await client.createTask(request);
            console.log("Task added to Queue", response.name);
            return { success: true };
          } catch (error) {
            console.log("Error While adding to queue", error);
            return { success: false };
          }
}

If you're wondering whether I'll be including any third-party APIs in the URL, the answer is no. I'll be doing some processing with the response data, not directly calling external APIs.

This will begin adding tasks to the queue while consistently adhering to the defined task rules.

If you're working with multiple third-party APIs, each having different rate limits, make sure to configure separate tasks with the required rate limits and add those to their respective queues.

That's all for today. If you spot any errors or inaccuracies, please don't hesitate to reach out. I'm always eager to learn and improve my knowledge. Thank you, and I'll see you in the next blog with an exciting topic!