Half of retailers say that the main reason that they haven’t adopted automated external data acquisition strategies is because web-scraping is too complex. A further 44% say they are also worried about legal complications.
While attempts have been made to educate organisations about the legal nuances of web scraping, there is still a need for the message to be communicated further across the industry.
So finds research by Oxylabs’ in its white paper, The Growing Importance of External Data in the Retail and E-Commerce Industry, goes on to highlight that 41% of respondents claimed their needs were already fully covered by other means of data collection, such as third-party databases. Purchasing third-party databases may redistribute some of the weight of legal compliance on the provider instead of the receiver.
Finally, 38% of respondents listed budget constraints and 35% listed lack of talent as reasons for not implementing web scraping. A large part of budget constraint issues can be mitigated by using out-of-the-box web scraping solutions. Managing these requires very few in-house employees but still promises effective data acquisition.
Julius Černiauskas Chief Executive Officer at Oxylabs comments: “At Oxylabs, the role of data has become even more significant in the last couple of years, with fewer businesses now relying exclusively on traditional sources to acquire this information. Therefore, as external data acquisition becomes useful, it is expected that organisations will increase their scraping budgets”.
Denas Grybauskas, the Head of Legal at Oxylabs adds: “However, the current web scraping landscape is burdened with a plethora of legal technicalities. A lack of industry regulation and ethics agreements makes the situation hugely complex. While many companies utilize web scraping on a daily basis, the best thing all organisations can do to avoid legal problems, other than consulting with a legal professional, is to only extract publicly available data. Trying to scrape other forms of data might seem harmless, but they are protected by a number of different laws which could lead to legal difficulties further down the line.”
Data strategies in 2022 and beyond
Over the next 12 months, 52% of e-commerce and retail organisations plan to collect more external data, with half looking into ways to collect more real-time data.
Currently, the most common reason retail and e-commerce organisations are using external data is to predict market trends (43% of businesses). The report also highlighted that 38% are benchmarking against competition and optimising customer service to help elevate the quality of service provided. Organisations view this approach as incredibly powerful, as it allows business development processes to gain a stronger foundation and build better roadmaps.
Černiauskas concludes: “It is clear to see that alternative data is playing, and will continue to play, an increasingly important role in the retail market. Whilst there are numerous incentives to acquire such data, the complex processes involved mean some companies are yet to make the most of it. With the right approaches, however, there is every possibility that businesses can take full advantage of web scraping.”
