[Proxy Scraper] Charon

[helyea]

Since I could not find a tool that allowed me to define custom sources that were not plaintext for scraping I decided to make my own.
It currently supports scraping from a plain-text source and from table elements using json presets.

Loading…

i.hawolt.com

How to create a new plain-scrape source
If you dont know what JSON is please make yourself familiar with it before proceeding

• https://www.impressivewebs.com/what-is-json-introduction-guide-for-beginners/

This example will show you how to create a configuration for proxyscrape.com

• Navigate to ?%userprofile%\.charon
• Create a new file
• The basic preset for your file looks like this
  

  {
  "plain":{
  "source":"",
  "delimiter":"",
  "break":",
  "type":""
  }
  }

  • source is the website to read from
  • delimiter is the key that is being used to split up each individual line to read ip and port seperately
  • break is the type of line-break used in this case it is \r\n
  • type is the type of proxy this source contains and it can either be https or socks in our case this will be https
• Save the file, it should look like this
  

  {
  "plain":{
  "source":"https://api.proxyscrape.com/?request=getproxies&proxytype=http&timeout=10000&country=all&ssl=all&anonymity=all",
  "delimiter":":",
  "break":"\r\n",
  "type":"https"
  }
  }

How to create a new table-scrape source
If you dont know what JSON is please make yourself familiar with it before proceeding

• https://www.impressivewebs.com/what-is-json-introduction-guide-for-beginners/

If you dont know what Jsoup is please make yourself familiar with it before proceeding

• https://jsoup.org/cookbook/extracting-data/selector-syntax
• https://try.jsoup.org/

This example will show you how to create a configuration for free-proxy-list.net

• Navigate to ?%userprofile%\.charon
• Create a new file
• The basic preset for your file looks like this
  

  {
  "table":{
  "source":"",
  "class":"",
  "index":,
  "siblings":[
  "",
  ...
  ""
  ],
  "type":"",
  "port":,
  "ip":
  }
  }

  • source is the website to read from
  • class is the class-name of the table element
  • index is the index of the previously fetched classes incase there is more than one table with the same name
  • siblings break down the element to the text source of the table
  • type is the type of proxy this source contains and it can either be https or socks in our case this will be https
  • port defines the column index that contains the port
  • ip defines the column index that contains the ip
• Save the file, it should look like this
  

  {
  "table":{
  "source":"https://free-proxy-list.net/",
  "class":"table-bordered",
  "index":0,
  "siblings":[
  "tbody",
  "tr",
  "td"
  ],
  "type":"https",
  "port":1,
  "ip":0
  }
  }

Download

Loading…

hawolt.com