Data analysis or analyst Terminology

 

🧠 I. Core Concepts & Processes

• 🔍 Data Analysis: The process of inspecting, cleaning, transforming, and modeling data to discover useful information, inform conclusions, and support decision-making.

• 🌐 Data Analytics: A broader term that encompasses the entire management of data, including analysis, tools, methods, and processes.

• 🧪 Data Science: An interdisciplinary field using scientific methods, algorithms, and systems to extract insights from data. It often involves advanced programming and statistical modeling.

• 💼 Business Intelligence (BI): The infrastructure for collecting, storing, and analyzing business data to optimize decision-making.

• 📊 Data-Driven Decision Making (DDDM): Basing decisions on data analysis rather than purely on intuition or observation.

---

📂 II. Types of Data

• 🧱 Structured Data: Organized in a predefined format (e.g., SQL databases, Excel rows/cols).

• 📄 Unstructured Data: No predefined format (e.g., emails, videos, social media posts).

• 🏷️ Semi-structured Data: Contains tags or markers but no formal structure (e.g., JSON, XML).

• 📏 Quantitative Data: Numerical data that can be measured (e.g., height, sales figures).

• 🎨 Qualitative Data: Descriptive, non-numerical data (e.g., interview transcripts, colors).

• 🐋 Big Data: Extremely large datasets characterized by the 3 V's (or 5 V's):

  • 📦 Volume: The sheer amount of data.

  • 🚀 Velocity: The speed of data generation/processing.

  • 🧩 Variety: The different types of data.

  • ✅ Veracity: The quality and accuracy.

  • 💎 Value: The usefulness of the data.

---

🗄️ III. Data Management & Storage

• 🛢️ Database: An organized collection of data stored electronically.

• 🗝️ SQL (Structured Query Language): The language used to communicate with databases.

• 🍃 NoSQL: Databases designed for unstructured data (e.g., MongoDB).

• 🏭 Data Warehouse: A central repository of integrated data used for reporting (e.g., Snowflake, BigQuery).

• 💧 Data Lake: A vast pool of raw data stored in its native format.

• 🔄 ETL (Extract, Transform, Load): Extracting data, transforming it, and loading it into a warehouse.

• 📥 ELT (Extract, Load, Transform): Loading data first, then transforming it within the target system.

• 🏪 Data Mart: A subset of a data warehouse dedicated to a specific team.

---

🧹 IV. Data Cleaning & Preparation

• 🧼 Data Cleansing/Wrangling: Detecting and correcting corrupt or inaccurate records. Often the most time-consuming step.

• ❓ Missing Data: Data points that are not recorded.

• 🧩 Imputation: Replacing missing data with substituted values.

• 📉 Outlier: A data point that differs significantly from others (can be an error or a finding).

• ⚖️ Normalization: Scaling numerical data to a standard range (e.g., 0 to 1).

• 📏 Standardization: Rescaling data to have a mean of 0 and a standard deviation of 1.

---

🧮 V. Statistics & Mathematics

• 📝 Descriptive Statistics: Summarizes the main features of a dataset.

  • 🎯 Mean: The average.

  • ↔️ Median: The middle value.

  • 🔁 Mode: The most frequent value.

  • 📶 Standard Deviation: Measure of variation/dispersion.

• 🕵️ Inferential Statistics: Using a sample to make inferences about a population.

  • 🌍 Population: The entire set.

  • 🧪 Sample: A subset of the population.

• ✅ Hypothesis Testing: Testing a hypothesis about a population using sample data.

• 🎲 P-value: Probability of results occurring by chance (≤ 0.05 usually means significant).

• 🔗 Correlation: A measure of the relationship between two variables (Correlation ≠ Causation).

• 📉 Regression Analysis: Estimating relationships between variables (e.g., Linear, Logistic).

---

🔬 VI. Data Analysis & Modeling

• 🔎 Exploratory Data Analysis (EDA): Analyzing datasets to summarize characteristics, often visually.

• 🩺 Diagnostic Analysis: Understanding why events happened.

• 🔮 Predictive Analysis: Using algorithms to identify the likelihood of future outcomes.

• 💊 Prescriptive Analysis: Recommending actions to affect outcomes.

• 🤖 Machine Learning (ML): Computers learning without explicit programming.

  • 👨‍🏫 Supervised Learning: Trained on labeled data.

  • 🕶️ Unsupervised Learning: Finding patterns in unlabeled data.

---

📊 VII. Data Visualization

• 🖥️ Dashboard: A visual display of key information on a single screen.

• 📐 Metric: A standard of measurement.

• 🎯 KPI (Key Performance Indicator): Measurable value demonstrating business objective success.

• 📈 Charts & Graphs:

  • Bar Chart: Compares categories.

  • Line Chart: Trends over time.

  • Histogram: Distribution of a variable.

  • Scatter Plot: Relationship between two variables.

  • Box Plot: Distribution based on quartiles.

---

👥 VIII. Roles & Responsibilities

• 🧐 Data Analyst: Interprets data to help make decisions (Reporting, Viz, Descriptive analysis).

• 🤝 Business Analyst: Bridges the gap between IT and business needs.

• 👨‍🔬 Data Scientist: Uses advanced ML/Stats to build predictive models.

• 👷 Data Engineer: Builds the infrastructure (pipelines, warehouses) for analysts.

• 🛠️ BI Developer: Specializes in designing dashboards and BI tools.

---

🛠️ IX. Tools & Technologies

• 🐍 Programming: Python (Pandas, NumPy), R.

• 📗 Spreadsheets: Excel, Google Sheets.

• 📊 BI Tools: Tableau, Power BI, Looker.

• 🛢️ Databases: SQL, PostgreSQL, MySQL.

• 🐘 Big Data: Hadoop, Spark, Kafka.

Data Warehouse Concepts

 

🏗️ Core Data Warehouse Concepts

• 🏢 Data Warehouse (DWH): A central repository of integrated, historical data designed for query and analysis. Its primary purpose is to support business intelligence.

• 🏪 Data Mart: A subset of a data warehouse tailored to serve a specific business line (e.g., Sales, Finance). It contains a focused collection of data.

• 🔄 Operational Data Store (ODS): A database for integrating data from multiple sources for operational reporting. It is more current and volatile, often acting as a staging area.

• 🌊 Data Lake: A vast repository holding massive amounts of raw data in its native format (structured and unstructured) until needed. No predefined schema required.

• 🏡 Data Lakehouse: A modern architecture combining the flexibility of data lakes with the data management and ACID transactions of data warehouses.

• 🚧 Staging Area: A temporary storage area for data extraction, cleansing, and transformation before loading into the warehouse. Not typically queried by end-users.

• 📊 Business Intelligence (BI): Technologies and practices for collecting, analyzing, and presenting business information to support decision-making.

---

📐 Data Modeling & Architecture

• 🗺️ Schema: The logical description of the entire database, including table structures and relationships.

• ⭐ Star Schema: The simplest schema consisting of a central fact table connected to multiple dimension tables in a star shape.

• ❄️ Snowflake Schema: A variation of the star schema where dimension tables are normalized into multiple related tables. Reduces redundancy but increases complexity.

• 🌌 Galaxy Schema (Fact Constellation): A complex schema with multiple fact tables sharing dimension tables.

• 🔐 Data Vault Modeling: A hybrid method for long-term historical storage, composed of Hubs, Links, and Satellites. Resilient to change and highly scalable.

• 🏷️ Dimension: A category of information (the "who, what, where, when"). Provides context to facts (e.g., Customer, Product).

• 🔢 Fact: A measurement or metric, typically numerical (e.g., Sales Amount, Quantity).

• 🟢 Fact Table: The central table containing measurements (facts) and foreign keys.

• 🔵 Dimension Table: A table storing descriptive attributes related to a business dimension (e.g., Customer Name, City).

• 🌾 Grain (Granularity): The level of detail in a fact table (e.g., "one row per line item").

• 🔑 Surrogate Key: A system-generated unique identifier (integer) used as a primary key, independent of the source system.

• 🆔 Natural Key (Business Key): An identifier from the operational source system (e.g., CustomerID).

• 🕰️ Slowly Changing Dimension (SCD): Techniques to manage data changes over time.

  • Type 1: Overwrite old value (No history).

  • Type 2: Add new row (Preserve history).

  • Type 3: Add new column (Limited history).

• 🤝 Conformed Dimension: A dimension that represents the same thing across different fact tables (e.g., Date).

---

🚀 ETL & Data Integration

• 🚚 ETL (Extract, Transform, Load): The process of moving data from source to warehouse.

  • Extract: Reading data.

  • Transform: Cleaning and structuring data.

  • Load: Writing data to the target.

• ☁️ ELT (Extract, Load, Transform): Loading data into the target system before transformation. Common in modern cloud platforms (Snowflake, BigQuery).

• 🚰 Data Pipeline: A system moving data from one place to another; may or may not involve heavy transformation.

• 📸 Change Data Capture (CDC): Identifying and capturing changes (inserts, updates, deletes) in a source database to apply them to the warehouse in near real-time.

• 🧹 Data Cleansing: Detecting and correcting corrupt or inaccurate records.

• 🔍 Data Profiling: Examining source data to collect statistics and assess quality.

• 🛡️ Data Governance: Managing data availability, usability, integrity, and security across an enterprise.

---

🎯 Key Performance Indicators & Metrics

• 📈 KPI (Key Performance Indicator): A measurable value demonstrating how effectively a company achieves objectives.

• 📏 Measure (Metric): A numerical value that can be aggregated.

• ➕ Additive Measure: Can be summed across all dimensions (e.g., Sales Amount).

• 🌗 Semi-Additive Measure: Can be summed across some dimensions but not all (e.g., Account Balance).

• 🚫 Non-Additive Measure: Cannot be summed (e.g., Ratios, Percentages).

---

🧊 OLAP & Querying

• 🧠 OLAP (Online Analytical Processing): Technology for interactive, multidimensional data analysis.

• 🧾 OLTP (Online Transactional Processing): Systems managing transaction-oriented applications (e.g., ERP, CRM).

• 📦 Cube: A multi-dimensional array of pre-aggregated data for fast querying.

• ↕️ Drill Down / Roll Up: Navigating data hierarchy from summary to detail (Drill Down) or detail to summary (Roll Up).

• 🍰 Slice and Dice: Viewing data from different perspectives by selecting subsets.

• 🔄 Pivot: Changing the dimensional orientation of a report.

• ⌨️ MDX / DAX: Query languages for OLAP cubes (MDX) and Power BI/Analysis Services (DAX).

---

☁️ Modern Cloud & Big Data Terminology

• 🕸️ Data Mesh: Decentralized architecture organizing data by business domains, treating data as a product.

• 🧵 Data Fabric: Architecture providing a unified layer for data management across disparate environments.

• 🖥️ Data Warehouse Appliances: Pre-configured hardware/software bundles (e.g., Teradata, Netezza).

• ⚡ MPP (Massively Parallel Processing): Multiple processors working simultaneously on a task; used by cloud DWHs like Snowflake.

• 🐊 Data Swamp: A deteriorated, unmanaged data lake with little value.

• 👻 Serverless: Cloud execution model where the provider manages machine resources dynamically (e.g., BigQuery).

---

📚 General & Administrative Terms

• 🏷️ Metadata: "Data about data." Describes structure, source, and characteristics.

• 👣 Data Lineage: Visual representation of data's origin and movement through systems.

• 📖 Data Catalog: Centralized inventory of data assets helping users find and understand data.

• 👑 Master Data Management (MDM): Managing critical data (customer, product) for a single point of reference.

• ✅ Data Quality: The accuracy, completeness, consistency, and timeliness of data.

• 📉 BI Tool: Software for creating reports/dashboards (Tableau, Power BI).

• ❓ Ad-hoc Query: A non-standard, one-time query created by a user.

• 💾 Materialized View: A pre-computed view stored physically to improve performance for complex queries.