Industrial organizations involved
Agricultural companies, using sensors for soil and meteorological factors monitoring. Companies working in the tourist sector – tour operators, event organizers.
Technical/scientific challenge
Reliable, fast, and accurate transferring of environmental (meteorological) data, followed by its computational analysis (descriptive and predictive), is a considerable technical challenge. On one hand, it is difficult to methodologically organize the large volumes of data being transferred, the enormous diversity of environmental sensors used, and the proliferation of the transmission protocols. On the other hand, consistent assurance and maintenance of collected data quality while selecting the right tooling and algorithms for data analytics and presentation, requires specialized knowledge and technological experience.
Solution
To solve those challenges, a referential IoT architecture is proposed, designed and implemented with the following components: data extraction and cleaning module NiFi, data distribution module Kafka, data storage module Hadoop HDFS, data analytics modules Hive, Impala and Hue, and data presentation module PowerBI. Building such an architecture enables flexibility and speed in data loading and clean up from a multitude of meteorological sensors, different networks and protocols, reliable ingestion, dispatch, consumption and storage of the extracted environmental physical values, powerful data analysis engines for both streaming and batch processing, for analysis and design of visual presentation of the results to be easily understood by all different types of users..
Business impact
At the moment, there is a lack of real-time analytics and predictive instruments for forecasting bad weather conditions, alerting for emergency situations and natural disasters. Real-time analytics of meteorological data could contribute to lowering risks, saving human lives and health, decreasing business losses, preventing unexpected and undesirable consequences.
The developed solution could support:
- Tourist companies, organizing tours and events
- Companies in the hotel industry sector – climate analysis for hotel chain expansion
- Companies in the agricultural sector – for business processes optimization (providing optimal conditions for growing crops), improving product quality, minimizing risks, etc.
Benefits
- Flexible and reliable, multi-protocol and multi-network data extraction
- Cost effective Open-Source components
- Distributed high availability file system and event dispatching system
- Powerful data analytics including streaming and batch processing
- Sophisticated visualisation