Empowering Big Data insights and ensuring reliability in data ecosystems for parallel processing
Industrial organisations involved: Discordia
Technical/scientific challenge: It was discovered that managing the growing complexity of datasets in logistics operations presented a significant challenge for maintaining high data quality. The organization generates and processes large-scale data from diverse sources, including vehicle tracking, operational schedules, customer orders, and financial records. These data streams are integral to ensuring timely deliveries, optimizing resource allocation, and supporting real-time decision-making. However, inconsistencies, incomplete records, and delayed updates often disrupted workflows, leading to inefficiencies and potential operational risks.
HPDA-as-a-Service as Cloud service for Financial services
Industrial organisations involved: Finalcial services providers, Easy Credit
Technical/scientific challenge: We’re addressing how to efficiently process and analyze large, complex, and often unknown datasets in financial services, where traditional serial processing methods are too slow and resource-intensive. This requires transforming data workflows into parallel processes that can scale, handle uneven data distribution, and optimize resource use across distributed systems like Spark and Hadoop. The goal is to ensure fast, reliable, and scalable data analysis while overcoming challenges like data partitioning, processing bottlenecks, and optimizing performance across nodes in a distributed environment.
Distribution of Data for Parallel Processing & Analysis for Financial Services
Industrial organisations involved: Easy Credit
Technical/scientific challenge: Designing a system that dynamically provisions a Hadoop cluster while seamlessly integrating it with a business intelligence and data analytics tool introduces a lot of technical and scientific challenges. These challenges span across several areas: system architecture design, robust data management, scalable performance capabilities and security and compliance requirements.
Revolutionizing Data Integrity: A High-Performance Data Analytics
Approach to Scalable Data Quality Management
Industrial organisations involved: Coca-Cola Hellenic
Technical/scientific challenge: Coca-Cola Hellenic needed to improve the accuracy, consistency, and reliability of data being processed across its diverse datasets from multiple sources (sales figures, production schedules, logistics data). The company required an advanced data quality management solution capable of handling large-scale data ingestion while minimizing resource usage and ensuring compliance with internal data governance standards.
Cloud-based HPC Simulations on Discoverer with Nimbix
Industrial organisations involved: Discoverer Petascale Supercomputer, Sofia Tech Park
Technical/scientific challenge: Running HPC simulations traditionally requires a bit of technical expertise. The users need to establish secure connections to the supercomputer, work in a Linux command-line terminal, upload and download data and results, prepare job scripts, work with the scheduling software, know which storage partitions to use and keep track of their resource quotas.
Augmented Big Data Quality as Initial Process of Supercomputer Task Development
Industrial organisations involved: Analytical companies, Cloud service providers offering platforms for cleaning and enriching experimental big data, Agricultural companies leveraging sensor technology for monitoring soil and meteorological factors
Technical/scientific challenge: Integration Complexities and Ensuring Data Reliability and Security in Supercomputer-driven Augmented Big Data Systems Using AI and HPC Technologies.
Getting insights on Customers data and probability of default with the help of AI algorithms
Industrial organisations involved: Financial Services, Retail
Technical/scientific challenge: The financial industry grapples with assessing the creditworthiness of customers. Traditional models rely on static data, ignoring dynamic factors affecting a customer’s financial behavior. The challenge is to create an AI-based system that continuously analyses customer big data, considering economic changes and personal spending habits, to predict the likelihood of default accurately.
ANALYZING EFFICIENCY OF AD CHANNELS FROM MULTIPLE PLATFORMS
Industrial organisations involved: Companies in the retail sector, Marketing companies, Software developers, Analytical Companies
Technical/scientific challenge:
In the evolving digital advertising landscape, brands and marketers invest across multiple platforms like Meta (formerly Facebook), Instagram, Google Ads, TikTok, LinkedIn, Snapchat, and more. Each of these platforms provides a plethora of data related to ad performance, audience engagement, and conversion metrics. However, integrating and comparing data from these disparate sources with their own specific APIs to derive a holistic view of advertising efficiency remains a significant challenge. Marketers need a comprehensive solution to assess and compare the ROI of each ad channel to optimize their marketing budget.
eNVIRONMENTAL DATA: FROM TRANSFERRING AND ANALYTICS TO DECISION MAKING
Industrial organisations involved: Agricultural companies, using sensors for soil and meteorological factors monitoring. Companies working in the tourist sector – tour operators, event organizers.
Technical/scientific challenge:
Reliable, fast, and accurate transferring of environmental (meteorological) data, followed by its computational analysis, is a considerable technical challenge. On one hand, it is difficult to methodologically organise the large volumes of data being transferred, the enormous diversity of environmental sensors used, and the proliferation of the transmission protocols. On the other hand, consistent assurance and maintenance of collected data quality while selecting the right tooling and algorithms for data analytics and presentation, requires specialised knowledge and technological experience.
TRAFFIC EVENTS AND ALERTS:
DATA INTEGRATION AND ANALYTICS TO SUPPORT MANAGERIAL DECISIONS
Industrial organisations involved:
Companies in the transport sector; Companies in the Logistics and Supply Chain Management sector
Technical/scientific challenge:
The considerable technical challenge is to accurately identify, extract, transfer and integrate reliable and fast-changing data from traffic events and alerts to be followed by sophisticated and exhaustive analysis (descriptive and predictive).