fbpx

Code: EDI-2020-18-UBIMET

Domain: Energy & Environment

Summary

Detecting anomalies in large datasets of weather forescast data.

Proposed by

 

UBIMET is the globally leading provider of meteorological forecast systems, information and tailor-made services. With all its services, UBIMET places the highest emphasis on quality and meeting the customers’ demands. Around the clock, 365 days a year, data, weather models, radar and satellite images are analysed and interpreted by high-performance computing systems and specially trained experts. The precise meteorological information provided by UBIMET enables companies to work faster, in a more efficient and cost-conscious manner, and more successfully in their sector than the competition.

Description

Anomaly detection is of interest in many industry areas. Thus, novel solutions on how to detect changes in large datasets is a challenge that requires efficient methods. While the problem can easily be transferred into other domains, this specific challenge focuses on weather forecast data.

 

Data

The challenge has the following sample datasets available for download

The sample data provided in the data catalogue contains 2 forecast model runs, each containing 320 parameters for 66 forecast hours to allow you to get familiar with the data format and parameters. The full data set will contain 2 historic runs per day with a forecast horizon of 66 hours for up to 2.5 years (~13 TB/year).

Expected outcomes

  • Identify: (1) datasets, (2) affected parameters and (3) time-step from which a deviation in their parameter characteristics from previous datasets (anomalies or step-changes) can be observed.
  • The system needs to be able to detect changes in near-real-time to inform subsequent systems of issues to prevent uncontrolled propagation. A live system will need to be able to process at least 150 MB/min with 320 parameters.
  • UBIMET, therefore encourages solutions that make use of scalable parallel algorithms and infrastructure.

How do we apply?

Follow by Email
Facebook
Twitter
YouTube
SOCIALICON