We scrape product data from suppliers and import them to our ecommerce and sync their prices and stock availability, and we do so even though our suppliers that fall into the scraping category do not provide XML or API but do have worthy products!

We use Octoparse to scrape product data from some of our suppliers that do not provide XML or API and import them to our ecommerce. We like the IP rotation feature because even the suppliers that provide XML or API block our requests, especially image links when we are trying to scrape them and download them, so IP rotation prevents us from this when we scrape data from users that do not provide XML/API.

The free plan is limited to low number of crawlers and concurrent runs as well as 10.000 records per export. It does not affect the companies though because companies like ours use the Standard or Professional plan.

The software is much easier to use, visually appealing, and on going customer support as well as tutorials have been created with the user in mind. Octoparse Web Scraper Experience: I have been looking professional web scraper for about two months now. I did try so many software's. Some was hidden mist! Most did not work at all. Then I did end up to get Octoparse web scraper! Wau! That cloud base software was exactly what I was looking for! This software really works. Software works even with some of the complex website. I definitely recommend! I use Octoparse on a daily basis and at my organization. there is no smoother way of web scraping! The software has never given me any issues. I think nobody can find better software to scrape data from web. Software It works exactly as expected. Octoparse is easy to use interface no experience scrapping websites is needed - but can do a lot. Octoparse software It has enabled me to ingest a large number of data point and focus my time on statistical analysis vs. data collection. It has safe me some much time! Same jobs would take me hours before and now data is collected in few minutes! When I need a quick way to grab structured web data, Octoparse software will be my first choice. It took time to learn the tool, but when you master it - there are lot's of powerful features. In just few days I managed to extract product information from thousands of products with very little effort! Using Octoparse to scrape a lot of data we needed was MUCH faster than custom building any solution. The user interface is intuitive, pricing very reasonable, and support was outstanding! To put it simply, if you've ever found a website where you wished that you could copy/paste hundreds of records from. Then Octoparse is your solution. It can automatically collect complete content structures such as product catalogs or search results. It's very user-friendly, yet sophisticated enough to extract data from highly dynamic websites. Octoparse software is data extraction tool that anyone can use to get data from the web. You'll never have to write a web scraper again and can easily create APIs from websites that don't have them. Octoparse software can handle interactive maps, calendars, search, forums, nested comments, infinite scrolling, authentication, dropdowns, forms, Javascript, Ajax and much more. Octoparse software is really the tool for your company!

easy to use.

Cost

Horrible company. I would not recommend anyone do business with them.

Nothing. They were unable to solve my business problem. I wanted to parse/scrape a particular website and the Octoparse service was unable to get past the Cloudflare anti-bot technology. I found a competing service that worked much better.

They were unable to solve my business problem. I wanted to parse/scrape a particular website and the Octoparse service was unable to get past the Cloudflare anti-bot technology. I found a competing service that worked much better. Additionally, they were unwilling to refund the charges made to my credit card, despite their claim they would remind me before making any charges (they did not send any reminders). I contacted them the day of the charges, and they still would not refund me. I had to contest the charges.

It took me about a day to look into all available web scrapers. At the end stopped on Octoparse for couple reasons. Pros: - Installs on Windows, so I could use spare Windows Server for scraping. No nodejs learning or programming needed. - GUI was simple to understand, can dump a list of links that need to be scraped, select content on the page that needs to go into Excel spreadsheet and click start. That's it, no need to select specific HTML divs or write regex code. Don't know how, but this was the only scraper that could analyze and grab a specific text on the page without setting any rules, the other scrapers I've tried had a hard time and had to make complicated rules. - During scraping opens the pages in a real browser, so Javascript, AJAX websites would work as well. - You can export to Excel, directly to SQL, MYSQ or Oracle database, CSV, TXT or HTML file. - You can also back up your scraped data to Octoparse as a backup, will be saved with your task. - Configuration and scraper apps run in different programs. If one suddenly would to shut down because of some error, other Octoparse tasks would still continue to work as nothing has happened. Cons: - Had a hard time adding a list of 50000 links into the queue, but not a problem because you can have multiple tasks 30-40K links in my case, just divide links between those tasks. - Did not say anywhere that it was saving the tasks to their servers, so that's why probably has trouble with large tasks. On the other hand, this one is also a Pro, because you can create tasks on your computer and load them up on your server just by restarting the app. Overall: You can have 2 active tasks running at the same time for free, if you want more, you can upgrade to a paid version. It takes about a second to open a page, so roughly you can scrape one page per second per task. Overall this worked better than great. Did not have to ask our devs to write a scraper, the time I spent creating the scraper would be the same amount of time I would spend discussing with our devs how to scrape the content. And now devs are asking me for stats on scraped data, not the other way around. If you do any marketing and wish to gather data for stats or just create your database from any website, super easy to do, recommend it.

When we were looking for an appropriate scraping software we tried every scraping program available on the market. After some days of testing it turned out that Octoparse is exactly that what we were looking for. This great and powerful tool completely outclasses competition in most of hard tasks: -it can load and pass through very complex and big websites -you can set more complex logic which is very useful even if rarely used -there are practically no websites that Octoparse can't load, doesn't matter on what base and system they were built -friendly environment, with easy to use GUI -great support with honest, and kind people. -and everything without a single line of script which is great! There are, of course, some cons too, because nothing is perfect and that's a normal thing. Sometimes program hangs on some sites. After some observations, it came out that it is not a website problem. It happens on sites that are built similar to for example Wikipedia, which construction is practically the same under every link. Probably it is not the API in workflow for designer too, because it is about simple: loop by list of URL -> extract data. Especially when there are more links. Sometimes it shows that site is still loading, sometimes just hangs on one opened page for hours to then just move on. I note that I used all program options, including advanced options too including timeout limit, reload web page and every other option. Nothing has helped. Finally, functionalities and enhancements that can be added in the future to Octoparse, to further improve it: Example situation: We add a 100 links to the program and 17 of them failed to execute. We found out that reload website option is unclear. If we e.g. use proxy and the connection will fail with it, we can now use other proxy or few proxies depending on settings. It will be great if i.e. one proxy will fail at connection in specified number of attempts (let's say 5-15 attempts) then Octoparse will delete this proxy or mark it as malfunctioning and avoid it in the future, still returning these 17 bad link in scores. And it will be even better if score list after export will have links that failed to execute, just mark them in a separate column, or add website status error code or something. This can help in retry to get data from the websites that failed to load.