Data Scientist Deutsche Telekom

We calculate with our models the fibre rollout strategic plan for  Telekom in whole Germany. With our models and data analytics we provide transparency and strategic insight. For a correct assesment of market opportunities it is necessary to get a a complete up-to-date picture of competitor activities. Therefore we need your help to build up a crawling pipeline, which will continously update available data about competitor activities from public sources

Activity description and concrete tasks:
The tasks comprise all activities to support the current crawling process and to improve this process continously, to get the latest data of competitor activities from public web sources in the most efficient way. Specifically there are three main activities. First for each web source, which hast to be analysed, a standard crawler script has to be adapted to the individual page layout of the web source. As a second task the crawling results should be collected and transfered to our database and a report system should be established to visualize changes due to the crawling reults in the database. The third task will take care of the automation of the crawling process. A git pipeline should be established, which makes an update of the adapted crawling scripts in an AWS-cloud environment. Apart of this the job includes new approaches to screen intelligently the German competitor market, understanding market strategies and provide transparencies to stakeholders