Big Data Analytics Tools

If your SQL tables are messy, your analytics will always lie to you. Data cleaning is not optional, it is the foundation of trustworthy insights. Here’s a simple breakdown of 13 essential SQL techniques every data engineer and analyst should know: 1. Replace NULL with a Default Value Use COALESCE to safely fill missing values during queries. 2. Delete Rows with NULL Values Remove incomplete records when they can’t be repaired. 3. Convert Text to Lowercase Standardize fields like names and emails for clean comparisons. 4. Find Duplicate Rows Identify values that appear more than once using GROUP BY. 5. Delete Duplicate Rows (Keep One) Remove duplicates while preserving a single valid entry. 6. Remove Leading & Trailing Spaces Trim whitespace so joins and comparisons don’t break. 7. Split Full Name into First & Last Extract components using SUBSTRING functions (simple cases only). 8. Standardize Date Formats Convert inconsistent date strings into a unified format. 9. Eliminate Special Characters Strip symbols while keeping alphanumeric data clean. 10. Identify Outliers Spot values outside expected upper/lower thresholds. 11. Remove Outliers Delete invalid or extreme values when necessary. 12. Fix Typo or Incorrect Values Correct inconsistent categories to avoid fragmentation. 13. Standardize Phone Number Format Keep only digits for clean, uniform phone fields. Messy data leads to messy decisions. Small SQL cleanup steps like these dramatically improve model accuracy, dashboards, and business reporting.

Data cleaning is a challenging task. Make it less tedious with Python! Here’s how to use Python to turn messy data into insights: 1. 𝗦𝘁𝗮𝗿𝘁 𝘄𝗶𝘁𝗵 𝗣𝗮𝗻𝗱𝗮𝘀: Pandas is your go-to library for data manipulation. Use it to load data, handle missing values, and perform transformations. Its simple syntax makes complex tasks easier.     2. 𝗛𝗮𝗻𝗱𝗹𝗲 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗗𝗮𝘁𝗮: Use Pandas functions like isnull(), fillna(), and dropna() to identify and manage missing values. Decide whether to fill gaps, interpolate data, or remove incomplete rows.     3. 𝗡𝗼𝗿𝗺𝗮𝗹𝗶𝘇𝗲 𝗮𝗻𝗱 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺: Clean up inconsistent data formats using Pandas and NumPy. Functions like str.lower(), pd.to_datetime(), and apply() help standardize and transform data efficiently.     4. 𝗗𝗲𝘁𝗲𝗰𝘁 𝗮𝗻𝗱 𝗥𝗲𝗺𝗼𝘃𝗲 𝗗𝘂𝗽𝗹𝗶𝗰𝗮𝘁𝗲𝘀: Ensure data integrity by removing duplicates with Pandas drop_duplicates() function. Identify unique records and maintain clean datasets.     5. 𝗥𝗲𝗴𝗲𝘅 𝗳𝗼𝗿 𝗧𝗲𝘅𝘁 𝗖𝗹𝗲𝗮𝗻𝗶𝗻𝗴: Use regular expressions (regex) to clean and standardize text data. Python’s re library and Pandas str.replace() function are perfect for removing unwanted characters and patterns.     6. 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗲 𝘄𝗶𝘁𝗵 𝗦𝗰𝗿𝗶𝗽𝘁𝘀: Write Python scripts to automate repetitive cleaning tasks. Automation saves time and ensures consistency across your data-cleaning processes.     7. 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗲 𝗬𝗼𝘂𝗿 𝗗𝗮𝘁𝗮: Always validate your cleaned data. Check for consistency and completeness. Use descriptive statistics and visualizations to confirm your data is ready for analysis.     8. 𝗗𝗼𝗰𝘂𝗺𝗲𝗻𝘁 𝗬𝗼𝘂𝗿 𝗖𝗹𝗲𝗮𝗻𝗶𝗻𝗴 𝗣𝗿𝗼𝗰𝗲𝘀𝘀: Keeping detailed records helps maintain transparency and allows others to understand your steps and reasoning. By using Python for data cleaning, you’ll enhance your efficiency, ensure data quality, and generate accurate insights. How do you handle data cleaning in your projects? ---------------- ♻️ Share if you find this post useful ➕ Follow for more daily insights on how to grow your career in the data field #dataanalytics #datascience #python #datacleaning #careergrowth

𝗗𝗼𝗻'𝘁 𝗷𝘂𝘀𝘁 𝗽𝗿𝗼𝗰𝗲𝘀𝘀 𝗺𝗮𝘀𝘀𝗶𝘃𝗲 𝗱𝗮𝘁𝗮. 𝗠𝗮𝘀𝘁𝗲𝗿 𝘁𝗵𝗲 𝗲𝗻𝗴𝗶𝗻𝗲𝘀. In a world generating 2.5 quintillion bytes daily, traditional databases can't keep up. Big data technologies power Netflix recommendations, Uber's pricing, and real-time fraud detection. Explore the Big Data Technologies to master for Data Engineers - 🎯 Your Learning Strategy: → Start with Spark (70% of job postings demand it) → Add Kafka for real-time streaming → Understand batch vs stream processing → Practice with real datasets—theory alone won't cut it ⚡ Core Technologies: → Hadoop/HDFS - Distributed storage foundation → Spark - 100x faster than MapReduce, handles batch + streaming + ML → Kafka - Real-time data streaming at scale → Hive/Presto - SQL on massive datasets 🔧 Essential Ecosystem: → Development: Jupyter, Docker, Git → Cloud: AWS EMR, Azure HDInsight, GCP Dataproc 📚 Top Resources: → Get started with Apache Spark - https://lnkd.in/d8bqkiGa → PySpark with Krish Naik- https://lnkd.in/dNqwptBA → SparkByExamples - https://lnkd.in/di87FHcU → Projects with Alex Ioannides, PhD - https://lnkd.in/dxhYZMJG → Tutorial by Databricks - https://lnkd.in/gaUZqNm5 → Learn Kafka with amazing tutorials by Confluent - https://lnkd.in/gRF_ZHVCMy 💡 Pro Tips: ✓ Understand data patterns before designing architecture ✓ Test with realistic volumes early ✓ Streaming is the future—invest time in Kafka + Spark Streaming Impact? Companies using big data tech are 5x faster at decisions, 6x more profitable. 💬 Which technology are you diving into first—Spark or Kafka?

Imagine you have 5 TB of data stored in Azure Data Lake Storage Gen2 — this data includes 500 million records and 100 columns, stored in a CSV format. Now, your business use case is simple: ✅ Fetch data for 1 specific city out of 100 cities ✅ Retrieve only 10 columns out of the 100 Assuming data is evenly distributed, that means: 📉 You only need 1% of the rows and 10% of the columns, 📦 Which is ~0.1% of the entire dataset, or roughly 5 GB. Now let’s run a query using Azure Synapse Analytics - Serverless SQL Pool. 🧨 Worst Case: If you're querying the raw CSV file without compression or partitioning, Synapse will scan the entire 5 TB. 💸 The cost is $5 per TB scanned, so you pay $25 for this query. That’s expensive for such a small slice of data! 🔧 Now, let’s optimize: ✅ Convert the data into Parquet format – a columnar storage file type 📉 This reduces your storage size to ~2 TB (or even less with Snappy compression) ✅ Partition the data by city, so that each city has its own folder Now when you run the query: You're only scanning 1 partition (1 city) → ~20 GB You only need 10 columns out of 100 → 10% of 20 GB = 2 GB 💰 Query cost? Just $0.01 💡 What did we apply? Column Pruning by using Parquet Row Pruning via Partitioning Compression to save storage and scan cost That’s 2500x cheaper than the original query! 👉 This is how knowing the internals of Azure’s big data services can drastically reduce cost and improve performance. #Azure #DataLake #AzureSynapse #BigData #DataEngineering #CloudOptimization #Parquet #Partitioning #CostSaving #ServerlessSQL

Unlocking the Secrets of Cloud Costs: Small Tweaks, Big Savings! Three fundamental drivers of cost: compute, storage, and outbound data transfer. 𝐂𝐨𝐬𝐭 𝐎𝐩𝐬 refer to the strategies and practices for managing, monitoring, and optimizing costs associated with running workloads and hosting applications on provider’s infrastructure. 𝐖𝐚𝐲𝐬 𝐭𝐨 𝐌𝐢𝐧𝐢𝐦𝐢𝐳𝐞 𝐂𝐥𝐨𝐮𝐝 𝐇𝐨𝐬𝐭𝐢𝐧𝐠 𝐂𝐨𝐬𝐭𝐬: 💡𝐑𝐢𝐠𝐡𝐭-𝐒𝐢𝐳𝐢𝐧𝐠 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬: 📌 Ensure you're using the right instance type and size. Cloud providers offer tools like Compute Optimizer to recommend the right instance size. 📌 Implement auto-scaling to automatically adjust your compute resources based on demand, ensuring you're only paying for the resources you need at any given time. 💡𝐔𝐬𝐞 𝐒𝐞𝐫𝐯𝐞𝐫𝐥𝐞𝐬𝐬 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞𝐬: 📌 Serverless solutions like AWS Lambda, Azure Functions, or Google Cloud Functions allow you to pay only for the execution time of your code, rather than paying for idle resources. 📌 Serverless APIs combined with functions can help minimize the need for expensive always-on infrastructure. 💡𝐔𝐭𝐢𝐥𝐢𝐳𝐞 𝐌𝐚𝐧𝐚𝐠𝐞𝐝 𝐒𝐞𝐫𝐯𝐢𝐜𝐞𝐬: 📌 If you're running containerized applications, services like AWS Fargate, Azure Container Instances, or Google Cloud Run abstract away the management of servers and allow you to pay for the exact resources your containers use. 📌 Use managed services like Amazon RDS, Azure SQL Database, or Google Cloud SQL to lower costs and reduce database management overhead. 💡𝐒𝐭𝐨𝐫𝐚𝐠𝐞 𝐂𝐨𝐬𝐭 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧: 📌 Use the appropriate storage tiers (Standard, Infrequent Access, Glacier, etc.) based on access patterns. For infrequently accessed data, consider cheaper options to save costs. 📌 Implement lifecycle policies to transition data to more cost-effective storage as it ages. 💡𝐋𝐞𝐯𝐞𝐫𝐚𝐠𝐞 𝐂𝐨𝐧𝐭𝐞𝐧𝐭 𝐃𝐞𝐥𝐢𝐯𝐞𝐫𝐲 𝐍𝐞𝐭𝐰𝐨𝐫𝐤𝐬 (𝐂𝐃𝐍𝐬): Using CDNs like Amazon CloudFront, Azure CDN, or Google Cloud CDN can reduce the load on your backend infrastructure and minimize data transfer costs by caching content closer to users. 💡𝐌𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠 𝐚𝐧𝐝 𝐀𝐥𝐞𝐫𝐭𝐬: Set up monitoring tools such as CloudWatch, Azure Monitor etc. to track resource usage and set up alerts when thresholds are exceeded. This can help you avoid unnecessary expenditures on over-provisioned resources. 💡𝐑𝐞𝐜𝐨𝐧𝐬𝐢𝐝𝐞𝐫 𝐌𝐮𝐥𝐭𝐢-𝐑𝐞𝐠𝐢𝐨𝐧 𝐃𝐞𝐩𝐥𝐨𝐲𝐦𝐞𝐧𝐭𝐬: Deploying applications across multiple regions increases data transfer costs. Evaluate if global deployment is necessary or if regional deployments will suffice, which can help save costs. 💡𝐓𝐚𝐤𝐞 𝐀𝐝𝐯𝐚𝐧𝐭𝐚𝐠𝐞 𝐨𝐟 𝐅𝐫𝐞𝐞 𝐓𝐢𝐞𝐫𝐬: Most cloud providers offer free-tier services for limited use. Amazon EC2, Azure Virtual Machines, and Google Compute Engine offer limited free usage each month. This is ideal for testing or running lightweight applications. #cloud #cloudproviders #cloudmanagement #costops #tech #costsavings

Many high-paying data engineering jobs require expertise with distributed data processing, usually Apache Spark. Distributed data processing systems are inherently complex; add to the fact that Spark provides us with multiple optimization features (knobs to use), and it becomes tricky to know what the right approach is. Trying to understand all of the components of Spark feels like fighting an uphill battle with no end in sight; there is always something else to learn or know about. What if you knew precisely how Apache Spark works internally and the optimization techniques that you can use? Distributed data processing system's optimization techniques (partitioning, clustering, sorting, data shuffling, join strategies, task parallelism, etc.) are like knobs, each with its tradeoffs. When it comes to gaining Spark (& most distributed data processing system) mastery, the fundamental ideas are: 1. Reduce the amount of data (think raw size) to be processed. 2. Reduce the amount of data that needs to be moved between executors in the Spark cluster (data shuffle). I recommend thinking about reducing data to be processed and shuffled in the following ways: 1. Data Storage: How you store your data dictates how much it needs to be processed. Does your query often use a column in its filter? Partition your data by that column. Ensure that your data uses file encoding (e.g., Parquet) to store and use metadata when processing. Co-locate data with bucketing to reduce data shuffle. If you need advanced features like time travel, schema evolution, etc., use table format (such as Delta Lake). 2. Data Processing: Filter before processing (Spark automatically does this with Lazy loading), analyze resource usage (with UI) to ensure maximum parallelism, know the type of code that will result in data shuffle, and identify how Spark performs joins internally to optimize its data shuffle. 3. Data Model: Know how to model your data for the types of queries to expect in a data warehouse. Analyze tradeoffs between pre-processing and data freshness to store data as one big table. 4. Query Planner: Use the query plan to check how Spark plans to process the data. Ensure metadata is up to date with statistical information about your data to help Spark choose the optimal way to process it. 5. Writing efficient queries: While Spark performs many optimizations under the hood, writing efficient queries is a key skill. Learn how to write code that is easily readable and able to perform necessary computations. Here is a visual representation (zoom in for details) of how the above concepts work together: ------------------- If you want to learn about the above topics in detail, watch out for my course “Efficient Data Processing in Spark,” which will be releasing soon! #dataengineering #datajobs #apachespark

!! Spark 4.0 !! The release of Spark 4.0 marks a significant milestone in big data analytics, bringing a suite of technical enhancements and features that will revolutionize your data workflows. Here's a deep dive into the major improvements: 🔹 Performance Enhancements: Catalyst Optimizer Upgrades: Improved query planning and optimization. Tungsten Execution Engine: Enhanced memory management and execution efficiency. 🔹 New APIs and Functions: DataFrame and Dataset APIs: New methods for better data manipulation and querying. Expanded SQL Functions: Additional functions and extended support for ANSI SQL standards. 🔹 Pandas Integration: Compatibility: Improved interoperability with Pandas DataFrames. Pandas UDFs: Vectorized operations for faster and more efficient user-defined functions. 🔹 Data Source Connectivity: New Connectors: Support for a wider range of cloud storage and databases. Improved Format Integration: Enhanced support for Parquet, ORC, Avro, and other formats. 🔹 Machine Learning Library (MLlib): Algorithmic Enhancements: Introduction of new algorithms and performance improvements. Framework Integration: Better integration with TensorFlow and PyTorch for advanced machine learning tasks. 🔹 Streaming and Structured Streaming: Real-Time Processing: New features for more efficient real-time data handling. Fault Tolerance: Enhanced mechanisms for fault tolerance and recovery. 🔹 Graph Processing with GraphX: New Algorithms: Latest graph algorithms and optimizations. API Improvements: Streamlined API for graph manipulations. 🔹 Security and Governance: Data Security: Enhanced encryption, authentication, and secure data transfer. Governance: Improved data lineage and compliance management. 🔹 Documentation and Usability: Updated Documentation: More comprehensive and user-friendly documentation. Debugging Tools: Enhanced error messages and debugging capabilities. 🔹 Python 3.10+ Compatibility: Language Support: Full support for Python 3.10 and newer versions, incorporating the latest language features. 🔹 Adaptive Query Execution (AQE): Dynamic Optimizations: Better handling of skewed data and runtime query plan adjustments. 🔹 Kubernetes Integration: Enhanced Support: Improved deployment and management of Spark clusters on Kubernetes. 🔹 Expanded Ecosystem Integration: Data Lakes and Warehouses: Better integration with various big data tools and platforms. #pyspark #bigdata #apachespark #dataengineering

How would you 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁𝗹𝘆 𝗽𝗿𝗼𝗰𝗲𝘀𝘀 𝗮 𝟱𝟬𝟬 𝗚𝗕 𝗱𝗮𝘁𝗮𝘀𝗲𝘁 𝗶𝗻 𝗣𝘆𝗦𝗽𝗮𝗿𝗸, and how would you 𝘀𝗶𝘇𝗲 𝘆𝗼𝘂𝗿 𝗰𝗹𝘂𝘀𝘁𝗲𝗿? 🔹 𝗦𝘁𝗲𝗽 𝟭: 𝗙𝗼𝗿𝗺𝗮𝘁 𝗙𝗶𝗿𝘀𝘁 • Convert raw data to efficient formats • Use #Parquet or Delta Lake instead of CSV/JSON to enable columnar storage, compression, and predicate pushdown — all of which speed up query execution. 🔹 𝗦𝘁𝗲𝗽 𝟮: 𝗣𝗮𝗿𝘁𝗶𝘁𝗶𝗼𝗻𝗶𝗻𝗴 𝗠𝗮𝘁𝗵 • Split data for parallelism* • Divide the 500 GB dataset into ~4,000 partitions of 128 MB each. This ensures optimal task distribution across your cluster and avoids skew or underutilization. 🔹 𝗦𝘁𝗲𝗽 𝟯: 𝗖𝗹𝘂𝘀𝘁𝗲𝗿 𝗦𝗶𝘇𝗶𝗻𝗴 • Balance compute and memory • A setup like 10 nodes × 8 cores × 32 GB RAM gives you ~17 waves of execution. This balances speed and cost while keeping memory pressure manageable. 🔹 𝗦𝘁𝗲𝗽 𝟰: 𝗠𝗲𝗺𝗼𝗿𝘆 𝗠𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁 • Plan for shuffle-heavy operations • Joins and aggregations can triple memory usage. If your tasks exceed available RAM, #Spark spills to disk — so SSDs and memory-aware planning are essential. 🔹 𝗦𝘁𝗲𝗽 𝟱: 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗧𝘄𝗲𝗮𝗸𝘀 • Fine-tune Spark configs • Enable adaptive execution, tune `spark.sql.shuffle.partitions`, use broadcast joins where possible, and load data incrementally to reduce overhead. #DataEngineering #PySpark #BigData #ApacheSpark #CloudComputing #ETL #SparkOptimization #ClusterSizing #MemoryManagement #PerformanceTuning

How We Saved $10,000/Year by Re-Architecting Our Azure Data Pipeline When you're building data pipelines, it’s easy to default to managed services for simplicity. But sometimes, managing part of your own stack is the smarter (and cheaper) move. Our Scenario We initially built our data pipelines using: Azure Data Factory (ADF) for ETL Azure Data Lake Storage (ADLS) for storing raw and processed data Power BI for reporting Our data sources were SAP, MySQL, and PostgreSQL, and as volumes increased, the costs started stacking up. The Problem High operational costs due to daily ADF pipeline runs Growing need for low-latency queries and faster dashboards Increasing costs for storage + transformation + querying in the Azure ecosystem The Solution: Customizing the Architecture We re-architected the pipeline using reserved Azure VMs to host: Apache Spark (for ETL and transformations) ClickHouse (as our analytical DB for blazing-fast queries) Metabase (for dashboarding and reporting) The Impact Saved over $10,000 per year by reducing pay-per-use costs Gained full control over Spark optimizations Improved query performance significantly Simplified BI stack with Metabase + ClickHouse This transformation showcases how the right architecture, rather than tool substitutions, can drive substantial cost efficiencies and performance enhancements in data engineering. #DataEngineering #CostOptimization #Spark #ClickHouse #Metabase #ETL #Architecture #Azure #BigData Sumit Mittal

Messy data can be intimidating, but with SQL, you can turn chaos into clarity. Here’s how to tackle messy datasets step by step: 1️⃣ Start with a Data Audit Before you dive in, explore the data: 1. Check for missing values. 2. Look for duplicates. 3. Identify inconsistent formats (e.g., date formats or text cases). Use queries like: 𝘚𝘌𝘓𝘌𝘊𝘛 * 𝘍𝘙𝘖𝘔 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦 𝘓𝘐𝘔𝘐𝘛 10;  2️⃣ Handle Missing Values Decide how to deal with nulls: 1. Replace with a default value: 𝘜𝘗𝘋𝘈𝘛𝘌 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦 ⁣ 𝘚𝘌𝘛 𝘤𝘰𝘭𝘶𝘮𝘯_𝘯𝘢𝘮𝘦 = '𝘥𝘦𝘧𝘢𝘶𝘭𝘵_𝘷𝘢𝘭𝘶𝘦' ⁣ 𝘞𝘏𝘌𝘙𝘌 𝘤𝘰𝘭𝘶𝘮𝘯_𝘯𝘢𝘮𝘦 𝘐𝘚 𝘕𝘜𝘓𝘓;  2. Remove incomplete rows: 𝘋𝘌𝘓𝘌𝘛𝘌 𝘍𝘙𝘖𝘔 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦 ⁣ 𝘞𝘏𝘌𝘙𝘌 𝘤𝘰𝘭𝘶𝘮𝘯_𝘯𝘢𝘮𝘦 𝘐𝘚 𝘕𝘜𝘓𝘓;  3️⃣ Standardize Data Formats Make your data consistent: For text: 𝘜𝘗𝘋𝘈𝘛𝘌 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦 ⁣ 𝘚𝘌𝘛 𝘤𝘰𝘭𝘶𝘮𝘯_𝘯𝘢𝘮𝘦 = 𝘓𝘖𝘞𝘌𝘙(𝘤𝘰𝘭𝘶𝘮𝘯_𝘯𝘢𝘮𝘦);  For dates: 𝘚𝘌𝘓𝘌𝘊𝘛 𝘛𝘖_𝘋𝘈𝘛𝘌(𝘤𝘰𝘭𝘶𝘮𝘯_𝘯𝘢𝘮𝘦, '𝘠𝘠𝘠𝘠-𝘔𝘔-𝘋𝘋') ⁣ 𝘍𝘙𝘖𝘔 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦;  4️⃣ Remove Duplicates Clean up repeated rows: 𝘋𝘌𝘓𝘌𝘛𝘌 𝘍𝘙𝘖𝘔 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦 ⁣ 𝘞𝘏𝘌𝘙𝘌 𝘪𝘥 𝘕𝘖𝘛 𝘐𝘕 ( ⁣  𝘚𝘌𝘓𝘌𝘊𝘛 𝘔𝘐𝘕(𝘪𝘥) ⁣  𝘍𝘙𝘖𝘔 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦 ⁣  𝘎𝘙𝘖𝘜𝘗 𝘉𝘠 𝘤𝘰𝘭𝘶𝘮𝘯_𝘯𝘢𝘮𝘦 ⁣ );  5️⃣ Create New Fields for Clarity Sometimes, messy data needs additional columns for analysis: 1. Split a full name into first and last name: 𝘚𝘌𝘓𝘌𝘊𝘛 𝘚𝘗𝘓𝘐𝘛_𝘗𝘈𝘙𝘛(𝘧𝘶𝘭𝘭_𝘯𝘢𝘮𝘦, ' ', 1) 𝘈𝘚 𝘧𝘪𝘳𝘴𝘵_𝘯𝘢𝘮𝘦, ⁣   𝘚𝘗𝘓𝘐𝘛_𝘗𝘈𝘙𝘛(𝘧𝘶𝘭𝘭_𝘯𝘢𝘮𝘦, ' ', 2) 𝘈𝘚 𝘭𝘢𝘴𝘵_𝘯𝘢𝘮𝘦 ⁣ 𝘍𝘙𝘖𝘔 𝘵𝘢𝘣𝘭𝘦_𝘯𝘢𝘮𝘦;  6️⃣ Document Your Cleaning Process Keep track of all your changes in comments or a separate file. This makes it easier to explain your process and replicate it. Pro Tip: Always back up your original data before making changes! Messy data is just a puzzle waiting to be solved. With SQL, you have the tools to organize and prepare it for meaningful analysis. P.S. What’s the trickiest data-cleaning problem you’ve faced? Let’s share tips below! -- 👋 I’m Dr. Jayen T., Dedicated to helping aspiring data analysts thrive in their careers.