Yavor Christov
In his work, Yavor Christov discusses reference architectures that can fetch/receive, process, and analyze big data. The author also considers and reviews components that can be used for creating such architectures. His studies also propose a method for evaluating such architectures, as well as methods for integrating the components. There is also verification of the proposed method and architecture.
In this dissertation, the author discusses the basic characteristics of one of the most significant sources of big data – the Internet of Things (IoT). The work provides an extensive survey and classification of reference architectures with their key functionalities. Together with this, existing modules and components that can serve to build architectures for big data processing and analysis are considered. An approach for the quantitative evaluation of such architectures is suggested. It uses weights and expert assessment values following the baseline criteria and presented features of the ISO 30141 standard. An example of the assessment of existing reference architectures from Microsoft, Cloudera, and Google is also presented.
In his thesis, Yavor Christov also shows how verification of a big data architecture can be performed. He provides results from testing the proposed architecture with three different application datasets – from transport, finance, and environmental monitoring (weather data). The authors’ approach, followed by the verification and analysis, gives clear guidelines for the efficient use of the proposed architecture for big data collection and processing. This is done with the available tools for big data processing (mainly open-source packages and software are used). Last but not least comes the flexibility and ease of working with the proposed architecture and modular components.
Nikolay Kaskatijski
In his work, Nikolay Kaskatijski discusses different data transfer protocols when working with the Internet of Things (IoT). The author focuses on their work with big data processing systems. In his research, the author explores the applicability and features of a number of modern communication protocols from the perspective of big data systems.
The object of this dissertation is the methods for communication from IoT to systems for processing, storage, and analysis of large amounts of data. In addition, Mr. Kaskatijski looks at data transmission and reception approaches, data processing, and integration, as well as a methodology for selecting data communication protocols. Tasks addressed by the author include an overview and description of new information transfer protocols, as well as the definition of criteria for evaluating the applicability of individual protocols.
In his work, Nikolay Kaskatijski compares the characteristics of some of the best-known big data platforms and draws attention to two of the most widespread approaches for acquiring and processing data from different sources – real-time data and batch data. The author proposes a methodology for selecting important criteria for a given big data system implementation. In such way a user can make an estimate of the level of applicability of a particular data communication protocol, which can be under consideration. The work examines how an IoT object transmits data over different protocols, where the amount of data transferred and the energy used per unit time is measured. Five different protocols (MQTT, CoAP, AMQP, DDS and HTTP) are tested with structured, semi-structured, and unstructured data. Two of the most widely used big data platforms – Apache NiFi and Apache Kafka, were used to verify the results. The work demonstrates how the bigger number of messages leads to a bottleneck when writing them to the Haddop distributed file system (HDFS). The use of a buffer to accumulate messages before writing them to a file is proposed, and its impact in regard to the message reception and storage system is studied.