Skip to content

Google results scraper - This is just a fun project not for production use ๐Ÿ˜œ

Notifications You must be signed in to change notification settings

ramasubbaiya/google-results-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

5 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Google Results Scraper

Google results scraper: I was a given a boredom task to google stuffs and fill up excel sheet at my work. An excel sheet with 100 rows and 10 diff search terms. So basically 100 query words * 10 must words => 1000 google searches entries.

That made me write this. It is not great project but it helped me to finish my work.

Useful commands

Install dependencies

npm install

Run the project

npm start

Query

/**
 * Search query format, make your own search queries array
 * and replace this variable at line 15.
 *
 * Search query is saved in input.json
 */
const searchQueries = [
  {
    query: "tesla",
    searchTerms: "",
    mustContainTerms: "investment",
    mustContainAtLeastOneTerm: "",
    shouldNotAppearTerms: "",
    shouldContainLinkToURL: "",
    resultsRelatedtedToURL: "",
  },
];

HTML

All google result html pages are saved under html-dump.

Output

Google results is saved in output.json

Troubleshooting

Google limits the no. of requests I was only able to do around 700 requests per day then I was geeting 429 Too Many Requests error.

About

Google results scraper - This is just a fun project not for production use ๐Ÿ˜œ

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published