Skip to main content

Integrating Metad Analytics Cloud with Doris

· 4 min read
Tiven Wang
Leader of XpertAI

Summary​

This article explains how to integrate Metad Analytics Cloud with a Doris cluster. The integration combines multidimensional modeling and metric management with the parallel computing capabilities of Doris's MPP architecture to provide a powerful analytics and query solution.

info

For instructions on deploying Metad Analytics Cloud and a Doris cluster locally, see the two with-doris Docker Compose configurations in Deploy a Docker cluster.

Background​

Apache Doris is an open-source storage and computing engine for large-scale data. It focuses on high-performance queries and real-time analytics, using distributed columnar storage to support parallel computing and horizontal scaling. Through its massively parallel processing (MPP) engine, a Doris cluster can execute complex queries over large datasets concurrently and return results in real time.

Integrating Metad Analytics Cloud with Doris makes full use of Doris's parallel computing and high-performance query capabilities to accelerate analysis and decision-making. It combines multidimensional modeling and metric management with Doris storage and compute, enabling more comprehensive data management and analysis while improving data availability and quality. The integration also simplifies data workflows and operations, improving processing efficiency and accuracy. Together, Metad Analytics Cloud and Doris provide organizations with a more powerful, efficient, and reliable analytics platform for deeper business insight and informed decisions.

Multidimensional Modeling​

Multidimensional modeling organizes and manages data-warehouse data efficiently by building dimension-centered models and structuring data into fact and dimension tables.

Modeling directly from source tables in business systems can significantly reduce business complexity and improve model flexibility.

  1. Simpler data models: Multidimensional modeling turns complex business data models into simpler dimensional models. It identifies business facts and dimensions and maps them to fact and dimension tables. This structure aligns more closely with how business users understand data, making it easier to query while reducing reliance on complex SQL.
  2. Focus on business metrics: Business metrics become measures in fact tables, while dimension attributes provide descriptive context. Users can focus on the metrics that matter without first understanding every relationship and join between source tables.
  3. Flexible analysis: Separating data into dimensions and measures lets users combine dimensions, slice data, and drill down from different perspectives. This flexibility supports faster exploration and more accurate, efficient decision-making.
  4. Redundancy and precomputation: Storing selected aggregates and precomputed metrics in fact and dimension tables can accelerate queries and reduce repeated access to source data. This lowers query complexity and improves response time.

Compared with traditional data-warehouse modeling in SQL, multidimensional modeling can make better use of Doris's parallel computing capabilities:

  1. Dimensional models: Fact tables store business metrics, while dimension tables store the attributes that describe them. Joining fact and dimension tables supports complex queries and aggregations with a model that remains simple to understand. Doris can process these joins in parallel and calculate accurate results quickly.
  2. Aggregation and summarization: Precomputing and storing commonly used aggregates speeds up queries. Doris can execute multiple aggregation operations concurrently, improving response times.
  3. Query optimization: Carefully defined dimensions, measures, aggregates, and summarization strategies improve query performance. Doris's query optimizer can choose an efficient execution plan based on the query and data distribution, taking full advantage of parallel processing.

With appropriate storage formats, dimensional models, aggregation strategies, and query optimization, multidimensional modeling can use Doris's parallel computing capabilities to deliver efficient analytics and queries. Compared with traditional SQL-based warehouse modeling, it is well suited to integration with Doris and can improve both data processing and analytical performance.

Deployment​

There are two ways to deploy Metad Analytics Cloud with a Doris cluster locally:

  • Install the integrated Metad Analytics Cloud and Doris edition. This is suitable for users who do not yet have a big-data platform and want a one-step installation for a quick start.
  • Install the standard edition of Metad Analytics Cloud. This is suitable for users who already operate a Doris platform and want to connect it through Metad Analytics Cloud's data-source management features.

For standard-edition installation, see Installation. For the integrated edition, see Deploy a Docker cluster: Integrate a Doris cluster.

Conclusion​

Building multidimensional models directly from source tables can reduce business complexity and improve flexibility. By simplifying data models, focusing on business metrics, supporting flexible analysis, and using redundancy and precomputation where appropriate, it helps users understand and query data more easily and analyze it faster.

Metad Analytics Cloud provides a free installation package that can be integrated with a Doris cluster. Users can combine Doris compute with multidimensional modeling and metric management to support efficient analysis and querying for their data needs.