Spidering Advertisment Market

Customer

Value can be added to a platform by combining information from several other platform sources, therefore offering the most comprehensive selection of advertisements. The information retrieval solution should search for structured data and consolidate the information so that the user can gain the most value from it. This solution can be used for a diverse range of content such as job or property advertisements, news, events, POIs or information from social networks.

Project

The internet has become our most meaningful source of information. No other medium can keep up with the pace and scope of the internet. The information however, is often distributed over several portals and is only availabe in unconsolidated forms, which hinders its usability for businesses.

Solution

Astina developed a split spider application with a high capacity Host-System that works with a desired number of clients, making it highly scaleable. Whilst the core of 'Abiada' concentrates on aquiring raw data, thanks to a Plug-in system, random replacement module installation the data can be further worked with according to individual requirements. This allows information from diverse sources to be brought together, prepared and analysed, so that it can finally be used in a consolidated form. This also allows for regular updated information from the source platform to be exported in the desired format.

• The entire contents automatically undergo duplicate testing and similarity anaylsis.
• Data from a third source sich as Geoinformation can also be seamlessly integrated.
• The clients simulate different users, operating systems and browers and disguise the origin of the spidering (spoofing).
• The client server architecture is high performing and scalable, in order to consolidate more terabytes of data daily.
• The configuration of the spider can be worked on over a webfront and with more than one administrator.