Types of Analytics
Serving for Analytics
Serving data for analytics is fundamentally about enabling discovery, exploration, and the visibility of key patterns. For a data engineer, the architecture required to serve this data depends entirely on the specific use case: is the goal to analyze historical trends, trigger immediate automated alerts, or provide features within a customer-facing application?
Analytics serving generally falls into three distinct categories, each with unique requirements for latency, concurrency, and user audience.
1. Business Analytics (Strategic & Historical)
Business analytics focuses on using historical and current data to drive strategic decisions. This combines statistical analysis with human judgment. The serving layer here typically supports three main workflows:
Dashboards: These provide a high-level view of core metrics (KPIs/OKRs), similar to a car dashboard. They are designed for quick consumption by executives and managers to monitor health.
Reports: Often generated from deep dives, reports are used to investigate specific phenomena (e.g., why returns increased for a specific product). These drive insights that are shared with stakeholders.
Ad Hoc Analysis: This is the exploratory phase where analysts use SQL, Python, or Excel to answer specific, often temporary questions. Successful ad hoc analysis often leads to the creation of permanent reports or dashboard metrics.
Engineering Considerations:
Batch vs. Stream: Business analytics is often served via batch processing from data warehouses or data lakes. However, the ingestion frequency sets the "speed limit" for downstream insights; you cannot serve hourly insights if you only ingest data daily.
Storage: Some BI tools (like Tableau or Power BI) extract and store data internally for speed, while others query the data warehouse directly to leverage the power of the underlying OLAP database.
2. Operational Analytics (Immediate Action)
The defining characteristic of operational analytics is time. While business analytics offers "actionable insights" over a longer horizon, operational analytics demands "immediate action."
Real-time Response: This data is used to address problems the moment they occur. A classic example is application monitoring, where a spike in server load triggers an automated scaling event or an alert to an engineer.
The Necessity of Action: Real-time data is a distraction if it does not lead to an action. If data is streaming but the organization only reviews it weekly, the architecture is mismatched to the use case.
Convergence: The line between business and operational analytics is blurring. As streaming technologies mature, organizations are moving toward "streaming-first" architectures where real-time data is available for immediate operations but also stored for historical analysis.
3. Embedded Analytics (Customer-Facing)
Embedded analytics involves serving data directly to external end-users within a product or application (Data Apps). This shifts analytics from an internal back-office function to a core product feature.
Examples: A smart thermostat app showing a user their real-time energy usage, or an e-commerce seller dashboard showing inventory levels.
High Performance Requirements: Unlike internal analysts who might tolerate a slow query, external users expect consumer-grade performance.
Low Data Latency: Users expect updates (like a resume upload) to be reflected immediately.
Fast Query Speeds: Dashboards must load in seconds to maintain user engagement.
High Concurrency: The system must handle thousands of users querying simultaneously, which often requires specialized real-time databases rather than standard data warehouses.
Last updated