Scraping URLs containing query parameters

If the URL you want to scrape contains query parameters, it is required to first URL-encode it. This is important because otherwise our API wouldn't be able to differentiate between query params used in Scraping Fish API call vs. query params you want to pass to the desired web page.

cURL
Python
JavaScript

curl --data-urlencode 'url=https://example.com?example=param&second=parameter' \
https://scraping.narf.ai/api/v1/?api_key=[your API key]

import requests
from urllib.parse import quote_plus

url = quote_plus("https://example.com?example=param&second=parameter")
api_key = "[your API key]"

print(requests.get(f"https://scraping.narf.ai/api/v1/?render_js=true&api_key={api_key}&url={url}").content)

import axios from "axios";

const url = encodeURIComponent("https://example.com?example=param&second=parameter")
const apiKey = "[your API key]"
    
const response = await axios.get(`https://scraping.narf.ai/api/v1/?render_js=true&api_key=${apiKey}&url=${url}`);
console.log(response.data);

info

It is a good practice to always encode the URL, regardless of using query params or not.

Below is a list of links to documentation for URL encoding methods in selected popular programming languages:

You can also use the form below for URL encoding: