Big Data Analytics

BigData solutions overview

• What challenges BigData addresses
• BigData components in a nutshell
• How a typical BigData solution looks like
• How BigData platform handle data – Blocks, Partitions and distributed processing

Cloud and On-Premises solutions for big data

• What do different clouds offer for BigData Solutions
• BigData Solutions On-Premisses
• Hybryd approach
• Non-Cloud-Specific solutions

Introduction to Hadoop

• What is Hadoop and how it handles BigData
• Hadoop components
• What is the Hadoop Data File System and how it works

• Working with Hive

  • Creating tables (and their types – Managed / Extrnal)
  • Popuplating data in Hive
  • Storage Types (Avro, Parquet, ORC) and when to choose which

Intoductions to NoSQL

• What is NoSQL and when to use it
• NoSQL families
• HBase data modeling concepts

• MongoDB

  • MongoDB model concepts
  • MongoDB architecture (Clusters, Sharding)
  • Qurying and Updating Data with MongoDB

• Elasticsearch and Kibana

  • The ELK Stack
  • What is a  lucene index
  • How to analyze data with Kibana

• Graph Databases and Neo4j

  • The Neo4j modeling concepts and technique
  • Querying and updating data with Neo4j

Data analytics tools and Methods

• How to approach data analytics
• Data analytics with R
• Data analytics with Python
• Tableau for data analysis and data science
• Data storytelling

Data Ingestion

• Types of sources
• Streaming with Kafka

Data preparation methods

• Handling missing data
• Identifying and handling outliers
• Preparaing data for machine learning

Data preparation tools

• Working with Python and pandas
• Working with Spark
• Overview of existing tools (Tableau Data Prep, Knime, Talend, Trifacta)

Working with PySpark

• Concepts and architecture
• Working with PySpark Core (RDD)
• Working with PySpark SQL (Dataframes)
• Working with Data Streaming
• Data preparation and data preparation for ML

Advanced SQL

• Why SQL is still the best tool for data analytics and data preparation
• Windows functions
• CTE
• Aggregations

Cloud databases (Synapse, RedShift, BigQuery, Snowflake, Presto)

• Overview of the different BigData DBs
• Synapse architecture and concepts
• RedShift architecture and concepts
• Snoflake concepts
• Presto (Starbase) architecture and conepts

• Basic Knowledge of Python, or experience with other programming languages
• SQL
• Hands on experience with Databases