Diastema: Data-driven Stack for Big Data Applications Management and Deployment

Most techniques for data processing run on standard infrastructure management systems, while large datasets is being increasingly generated. The main challenge refers to using technology to gather efficient and faster insights from a dataset, considering not asking what data is easily obtainable and which tools are amenable to working with that dataset, but rather what question the analysis is trying to answer. This creates a landscape with data-intensive projects that prioritise technical prowess of execution over the robustness of analytical findings. Hence, a data-driven stack for big data applications management and deployment is being described, diastema, bringing efficient data-as-a-service data management through distributed storage and analytics, aiming at high performance and utilisation of heterogeneous resources, including abstraction, gateways, and small-footprint virtual machines. Diastema is evaluated through training a customer forecasting model for indicating customers behaviour, turning limited-value raw data to timely, relevant data, targeting at business agility and competitiveness.