Home > Technical & Creative Skills > Programming, Servers & Scripts

Help Scraping Suggestions from Search Box (7)

03-15-2019 11:06 PM #1 lorddevi (Member)
Help Scraping Suggestions from Search Box

Hello,

So I have this URL which has a typical search input box kind of thing in it. It allows you to type into it, and it will provide you suggestions or guesses as to what you are searching for.

I am very interested in having this data scraped, but have no idea where to go or how to get it done. I am hoping someone here might be able to help point me in the right direction.

Ideally, the right direction would be finding someone who would be willing to code a scraper of some kind for the data to be found here for us.

I would post it as a job even, but I am not sure I know where to even begin looking or what phrasing to use.


Basically, we want to scrape as many business names from the suggest drop down as possible.

Kind of like scraping YouTube, or Google, or Bing, or Amazon by typing "a", "aa", "b", "ba", etc etc into the search box for keyword generation.

But instead of keyword scraping, we are scraping the names and locations of various businesses in this input boxes's database.

The trick here is that it requires a minimum of 4 characters to be typed into the input box first before it will suggest anything. Likely to help prevent scraping just like I am looking to get done.

If you type in any random US city, and then type in like: "company", "best", "service", "dent", "auto" you will see all kinds of businesses pop up into the suggest dropdown.

I want as many of those as I can get from there.

Can anyone help guide me in the right direction? Perhaps a reader of this post who can create such a scraper, would be willing to help out for a fee?

DIRECT LINK to the Search Box in question.

Thanks guys!
-LD

03-16-2019 08:10 AM #2 gunnar (Member)

Hi LD,

if you look at the Inspector while searching you can see the request URL where the json data is fetched:

https://app.joinhomebase.com/onboard...york%20&q=abcd

now you can get a programmer that goes through a list of cities and tries the query alphabetical. Should be done in half an hour or so.

cheers and have good WE

03-18-2019 08:03 PM #3 lorddevi (Member)

Hi Gunmar,

Ok, that gets me closer to the mark. Doesn't sound like it should be too hard, which is very encouraging. Although it will have to use words in place of usual alphabet prefixes.

Thank you very much for your help.

Off to try and locate myself a programmer I guess!

03-18-2019 08:10 PM #4 twinaxe (Senior Moderator)

When you can't code yourself it should be easy doable with Zennoposter

03-19-2019 08:06 AM #5 vortex (Senior Moderator)

Quote Originally Posted by twinaxe View Post
When you can't code yourself it should be easy doable with Zennoposter
Was just going to suggest Zennoposter! Very easy to use and scrapes like a maniac.

I've used a few scrapers in the past - would definitely suggest to explore commercially-available scrapers before resorting to hiring someone to build a custom-made one.

For a list of scraper go to:

https://marketingtools.net/

Select 'scraper-tools' in the tag filter.




Amy

03-21-2019 06:56 PM #6 lorddevi (Member)

Oh great! Zennoposter, AND a new site for me explore!

lol, marketingtools.net looks pretty amazing! What a huge list of neat stuff to look at.

Thank you guys for the valuable information. This has been very helpful.

03-22-2019 08:06 AM #7 vortex (Senior Moderator)

Quote Originally Posted by lorddevi View Post
Oh great! Zennoposter, AND a new site for me explore!

lol, marketingtools.net looks pretty amazing! What a huge list of neat stuff to look at.

Thank you guys for the valuable information. This has been very helpful.
Haha yeah that list is amazing indeed! I had to pry myself away - would take days to check out all the tools if I'd indulged myself.



Amy

Home > Technical & Creative Skills > Programming, Servers & Scripts