How to use proxy with Axios

In our modern connected digital environment, privacy and security are top priorities. If you want to take full advantage of the potential of Axios, the famous JavaScript library for sending HTTP requests, then you are in the right place. In this blog post, we delve into the world of proxies and uncover the secrets to their easy integration into Axios.

Prerequisites

Axios is a JavaScript library, so we will use NodeJS for web scraping.

First, make sure NodeJS and npm are installed. Then create a new folder for the JavaScript files and initialize a new project with:

mkdir scrapeaxios
cd scrapeaxios
npm init -y

Note: The package.json file will be created with default values.

Next, install Axios and its dependencies.

npm i axios

To use a proxy with Axios

• Let’s look at an example of setting a proxy in Axios using HTTPBin as a target page and an IP from Proxy-cheap.

IP: '122.158.135.190',
port: 8080

• Import Axios and add the proxy details.

const axios = require('axios');
 
 
axios.get('https://httpbin.org/ip',
    {
        proxy: {
            protocol: 'http',
            host: '122.158.135.190',
            port: 8080,
        },
    }
)
    .then(res => {
        console.log(res.data)
    }).catch(err => console.error(err))

• Your code should return the IP address of your proxy since the Httpbin endpoint returns the public IP address of the requesting client.
• Run the code, and your result should look like this:

{ origin: '122.158.135.190' }

• Great job!

The JSON format of the response will then be useful. Here are a few methods that you can use to accomplish it:

1. Parse the response data using the JSON.parse method. You’ll get a JSON response if you succeed. Otherwise, it produces a string in most cases.

const axios = require('axios');


axios.get('https://httpbin.org/ip',
    {
        proxy: {
            protocol: 'http',
            host: '122.158.135.190',
            port: 8080,
        },
    }
)
    .then(res => {
        let data;
        try {
          data = JSON.parse(res.data);
        } catch (err) {
          // Use the response exactly as it is if it is not JSON.
          data = res.data;
        }
        console.log(data);
      }).catch(err => console.error(err))

1. Automatically parse the data using the responseType option.

const axios = require('axios');


axios.get('https://httpbin.org/ip',
    {
        proxy: {
            protocol: 'http',
            host: '122.158.135.190',
            port: 8080,
        },

        responseType: 'json',
    }
)
    .then(res => {
        console.log(res.data);
      }).catch(err => console.error(err))

• The code for both methods yields the same result as in the example above. However, method 1, without the try-catch block, would produce a syntax error. That’s because the response returned by HTTPBin is already in JSON format.

Proxy authentication with Axios: username and password

Paid proxies typically require a username and password for authentication. Therefore, Axios provides an authentication property that allows you to use a proxy server through this property.

auth: {
     username: '',
     password: ''
}

• To add properties to the script:

const axios = require('axios');


axios.get('https://httpbin.org/ip',
    {
        proxy: {
            protocol: 'http',
            host: '122.158.135.190',
            port: 8080,
                username: '',
                password: ''

        },

        responseType: 'json',
    }
)
    .then(res => {
        console.log(res.data);
      }).catch(err => console.error(err))

Rotate IPs with Proxy-cheap.

Rotating proxies use multiple pools of proxies and order each request randomly.

Let’s create the proxy rotator.

• First, define the proxy list array.

const axios = require('axios');

const proxyList = [
  { ip: '149.129.149.170', port: '8080' },
  { ip: '122.158.135.190', port: '5555' },
  { ip: '135.122.281.142', port: '8080' },
]; // Replace with your own list of proxies

• Then create a function that iterates over the list of proxies. For each request, it switches to the next proxy while sending the current proxy back to the end of the queue.

// Rotational function for the list of proxies
const rotateProxy = () => {
  const proxy = proxyList.shift(); // Get the following accessible proxy
  proxyList.push(proxy); // Removing the current proxy from the list and adding it back
  return {
    protocol: "http",
    host: proxy.ip,
    port: proxy.port,
  };
}

• Make Axios requests using the rotateProxy() function as the proxy option.

axios.get('https://httpbin.org/ip',
        {
            proxy: rotateProxy(),

        }
    )
        .then(res => {
            console.log(res.data);
        }).catch(err => console.error(err))

• If we put everything together and use a FOR LOOP to make multiple requests, our full code looks like this:

const axios = require('axios');

const proxyList = [
  { ip: '149.129.149.170', port: '8080' },
  { ip: '122.158.135.190', port: '5555' },
  { ip: '135.122.281.142', port: '8080' },
]; // Replace with your own list of proxies

// Rotational function for the list of proxies
const rotateProxy = () => {
  const proxy = proxyList.shift(); // Get the following accessible proxy
  proxyList.push(proxy); // Removing the current proxy from the list and adding it back
  return {
    protocol: "http",
    host: proxy.ip,
    port: proxy.port,
  };
}


for (let i = 0; i < 3; i++) {
    axios.get('https://httpbin.org/ip',
        {
            proxy: rotateProxy(),

        }
    )
        .then(res => {
            console.log(res.data);
        }).catch(err => console.error(err))
}

• Here is the result:

{ origin: '149.129.149.170' }
{ origin: '122.158.135.190' }
{ origin: '135.122.281.142' }

You are done!

Conclusion

In conclusion, integrating proxies with Axios opens up a world of possibilities for improving the security and performance of web applications. Armed with this newfound knowledge, you can confidently navigate the digital realm, protecting your data while ensuring seamless communication between applications and external services.