JavaScript 渲染

如果你打算抓取的页面需要执行 JavaScript 来动态加载所有必要的数据，你可以启用 JavaScript 渲染，而无需使用无头浏览器。

接收 HTML 格式的渲染页面内容

curl -k -v -x unblock.smartdaili-china.com:60000 \
-U "USERNAME:PASSWORD" "https://ip.smartdaili-china.com/" \
-H "X-SU-Headless: html"

import requests

proxies = {
    'http': 'http://USERNAME:[email protected]:60000',
    'https': 'http://USERNAME:[email protected]:60000'
}

headers= {
   "X-SU-Headless": "html"
}

response = requests.request(
    'GET',
    'https://ip.smartproxy.com/',
    verify=False,
    proxies=proxies,
    headers=headers
)

print(response.text)

import fetch from 'node-fetch';
import createHttpsProxyAgent from 'https-proxy-agent'

const username = 'YOUR_USERNAME';
const password = 'YOUR_PASSWORD';

const agent = createHttpsProxyAgent(
  `http://${username}:${password}@unblock.smartdaili-china.com:60000`
);

process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;

const headers = {
 	'X-SU-Headless': 'html'
}

const response = await fetch('https://ip.smartproxy.com/', {
  method: 'get',
  headers: headers,
  agent: agent
});

console.log(await response.text());

接收 PNG 格式的渲染页面内容

响应将包含可以以 PNG 格式保存的图像的原始字节

curl -k -v -x unblock.smartdaili-china.com:60000 \
-U "USERNAME:PASSWORD" "https://ip.smartdaili-china.com/" \
-H "X-SU-Headless: png"

import requests

proxies = {
    'http': 'http://USERNAME:[email protected]:60000',
    'https': 'http://USERNAME:[email protected]:60000'
}

headers= {
   "X-SU-Headless": "png"
}

response = requests.request(
    'GET',
    'https://ip.smartproxy.com/',
    verify=False,
    proxies=proxies,
    headers=headers
)

print(response.text)

import fetch from 'node-fetch';
import createHttpsProxyAgent from 'https-proxy-agent'

const username = 'YOUR_USERNAME';
const password = 'YOUR_PASSWORD';

const agent = createHttpsProxyAgent(
  `http://${username}:${password}@unblock.smartdaili-china.com:60000`
);

process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;

const headers = {
 	'X-SU-Headless': 'png'
}

const response = await fetch('https://ip.smartproxy.com/', {
  method: 'get',
  headers: headers,
  agent: agent
});

console.log(await response.text());