Web Scraping and Automation
Hire me for your next project, I have skills in Javascript (Puppeteer), and Python (Selenium). Whether you need to collect your competitor's pricing and automatically adjust yours or if you need to automate a task on the web, I can build you a bot which can save you hundreds or even thousands of human work hours.
Who is this for?
This is for someone or a company that is spending too much time on a mundane task on the web and needs to free up those man hours. Alternatively, if you need to collect loads of data accurately then it may make sense to automate the task.
FAQ And General Information
For collecting data it is always better to make use of lighter-weight packages like beautiful soup or cheerio. These types of parsers don't need to download the resources on a webpage (images, external javascript, CSS), only the raw HTML to get data. However not every website is scrapeable using these sorts of frameworks. For more complex websites browser-based frameworks like Puppeteer or Selenium would need to be used.
For automating tasks most projects will need to use browser-based frameworks Puppeteer or Selenium.
There are differences in how each framework works but for the most part, they are on par in terms of functionality. Puppeteer has better tooling for testing and has slightly better performance. I personally much rather use Selenium for 2 reasons: I can develop and easily test my bot in Jupiter notebook and secondly, most data science tools are available in python so it makes Selenium my go-to. But if your application is already in Javascript (node js) then I can build a bot with Puppeteer as well.
It all depends on how much work needs to be done. I charge R400 per hour so build a bot could take a single week (R25200) or it could take 3 weeks (R75600). After that, I would offer a retainer to maintain the project in case the website gets updated and makes the scraper no longer work, this retainer is purely optional but may make sense to take advantage of, the retainer could be anywhere from R500 to R1000 per month depending on how large the project is. The cost of building the bot is negligible if you consider that you would have to employ someone full-time to perform the task that the bot can do 24/7.
Ready to get started?