Decoding the drive

Revving up transformative data solutions in the automotive industry

Decoding the drive

Revving Up Transformative data solutions in the Automative industry

THE BIG PICTURE

Navigating data highways with efficient decoding Grasping petabytes of intricate data for informed decision-making Steering through the vast expanses of the automotive industry, our client is a vanguard in vehicle testing. Each test drive not only propelled their cars ahead but also amassed petabytes of intricate data. These nanosecond-level precision records paint each car's every move, from the tire's rotation to the windows opening and closing. But the roadblock? Decoding this data trove quickly and affordably.

THE CHALLENGE

1. The road bumps of automotive data Decoding specific data entities within 24 hours for post-test insights Every vehicle test conducted by our client resulted in terabytes of data daily. Through Electronic Control Unit (ECU) communications, this data offered intricate insights across thousands of data points, such as the vehicle's speed, movement, & more - the sheer volume of this information called for an efficient decoding method. The client's test drives generated several terabytes of data daily, adding to petabytes for multiple vehicles. The decoding of this enormous amount of data was costly and time-consuming. The client needed to retrieve specific data entities within 24 hours of the test runs, but they faced difficulties achieving that.

THE CHALLENGE

2. The high toll of processing costs & delays The data collected by the client is stored in a

standardized industry format called Measurement Data Format (MDF). This encoding format is designed for efficient and secure binary data storage. Decoding such a complex data structure, especially with the high precision required for the various signals, posed a significant challenge for the client. The company incurred an operational cost of 1,000 euros per TB to process. Additionally, the delays in data processing led to longer detection times for anomalies/failures, which in turn led to increased operational expenses.

SOLUTION

Shifting modes to scalable and

cost-effective decoding To address these challenges, Fractal used a tailored approach that harnessed the strengths of modern technologies and strategies to create a cost-effective, high-performance decoding solution called DataLogger, which we implemented across various domains within the organization. Our data engineering expertise and ability to quickly grasp domain-specific nuances resulted in an MVP that exceeded expectations. The solution was built to demonstrate the power of modernized scalable architecture leveraging the client-preferred Microsoft tech stack.

THE TRANSFORMATIVE SOLUTION

Fractal's DataLogger decodes physical streams for actionable insights By decoding signal data into physical streams, Fractal's Datalogger offers engineers insights into vehicle health, driver behavior, necessary pre-production corrections, and preemptive measures for electric vehicles, such as battery health assessments. This is achieved through a logical, nine-step process:

MVP Development & Signal Conversion Input: Test data from CAN Bus, several measurement data files. Process: Initial conversion of signals from CAN Bus

Deep Dive into Communication Protocols & Signal Recording Input: Raw data (ECU communications, Binary Data formats, protocols, reference AutoSAR XML files) Process: Thorough analysis of core communication protocols

Data Ingestion & Initial Processing Input: Terabytes of vehicle test data Process: Preliminary dry runs & preparation for extensive I/O operations

Big Data Integration and Auto-Scaling Input: Varying incoming data Process: Integration with Big Data tools; auto-scaling based on data load

THE TRANSFORMATIVE SOLUTION

Multithreaded Data Conversion & Writing Input: Continuous data stream. Process: One thread reads and converts, & another writes the output to Delta Lake storage. 6 7

Core Logic Enhancement and Optimization Input: Data ready for conversion. Process: Transition of core conversion logic to C++ for optimization.

Event-Driven Processing and Cloud Agnosticism Input:

Measurement data files from test drives. Process:

Use of Kubernetes' event-driven architecture for efficient scaling and parallel processing.

Operational Metadata Logging and Flexible Orchestration Input: Runtime data operations. Process: Use of serverless function APIs for logging; adaptable architecture for orchestration. 8 9

Output Delivery Output:

Decoded data in human-readable format, ready for business integration.

THE OUTCOME

From 0 to 100 with ‘the In-house Converter’

A leap in cost efficiency from 1,000 euros to 25 euros per TB

Performance Unleashed: With Fractal's solution, 1 TB of data was processed in less than an hour, a task that previously took an entire day. Cost Milestone: A massive leap in cost-efficiency was achieved, reducing the expense from 1,000 euros per TB to 25 euros per TB. In-House Mastery: Freed from external licensing costs, the client proudly named their new tool, the In-House-Converter (IHC) - celebrated within IT & Business teams.

40x Cost Reduction per TB

<60 minutes Time to process 1TB

Fractal’s businesses include Crux Intelligence (AI driven business intelligence), Eugenie.ai (AI for sustainability), Asper.ai (AI for revenue growth management) and Senseforth.ai (conversational AI for sales and customer service). Fractal incubated Qure.ai, a leading player in healthcare AI for detecting Tuberculosis and Lung cancer. Fractal currently has 4000+ employees across 16 global locations, including the United States, UK, Ukraine, India, Singapore, and Australia. Fractal has been recognized as ‘Great Workplace’ and ‘India’s Best Workplaces for Women’ in the top 100 (large) category by The Great Place to Work® Institute; featured as a leader in Customer Analytics Service Providers Wave™ 2021, Computer Vision Consultancies Wave™ 2020 & Specialized Insights Service Providers Wave™ 2020 by Forrester Research Inc., a leader in Analytics & AI Services Specialists Peak Matrix 2022 by Everest Group and recognized as an & ‘Honorable Vendor’ in 2022 Magic Quadrant ™ for data & analytics by Gartner Inc. For more information, visit fractal.ai Fractal is one of the most prominent providers of Artificial Intelligence to Fortune 500® companies. Fractal’s vision is to power every human decision in the enterprise, and bring AI, engineering, and design to help the world’s most admired companies. About Fractal

Corporate Headquarters Suite 76J, One World Trade Center, New York, NY 10007

Drive the change with smarter solutions

Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Page 8 Page 9

Made with FlippingBook - PDF hosting