Category
Instructor
Times Analytics
Data Engineering with GenAi
- Description
- Curriculum
- FAQ
Data engineering is a software engineering discipline focused on designing, building, and maintaining the systems and infrastructure that collect, transform, and store data—making it usable for analytics, reporting, and machine learning. It turns raw and often messy data into clean, reliable, and accessible information so organizations can derive actionable insights.
A Curriculum that prepares you to thrive in the Industry as an Expert Big Data EngineerDownload Syllabus |
Who Should Do Data Engineering Course?
🎯 Why Data Engineering Matters
Businesses generate vast amounts of data every day. Without proper engineering, this data remains unused or becomes a liability. Data engineering enables:
-
📈 Informed business decisions through data readiness
-
⚙️ Automation of repetitive data processes
-
🔐 Secure and compliant data storage
-
💡 Real-time insights and predictive analytics
Topics You will Master
-
Data engineering course
-
Data pipeline development
-
Big data architecture
-
Learn data engineering online
-
Real-time data processing
-
Data engineer training
Roles a Data Engineer Play?
Big Data Engineer |
Data Architect |
Business Intelligence Engineer |
Data Warehouse Engineer |
Technical Architect |
New Batch launch
| Course name | WeekEnd Batch | WeekDay Batch |
|---|---|---|
Data Engineer |
06-Sep- 202507:00 PM – 10:00 PM IST |
04-Sep- 202507:00 PM – 8:30 PM IST |
Classroom Training
Our comprehensive classroom program features:
- Lectures by certified Machine Learning professionals
- Hands-on training sessions for practical understanding
- Interactive learning environment with peer collaboration
- Unlimited doubt-clarification sessions
- Real-world project experience
Perfect for those who thrive in structured, face-to-face learning environments.
Corporate Training
We offer customized training solutions for businesses:
- Tailored curriculum aligned with your company goals
- Flexible scheduling to minimize workflow disruption
- Team-based learning approaches
- Progress tracking and reporting
- Ongoing support and consultation
Transform your workforce with skills that drive business results.
Online Training
Our virtual learning experience includes:
- Live interactive sessions with expert instructors
- Comprehensive curriculum covering Python to advanced ML algorithms
- Recorded sessions for flexible review
- Virtual lab environment for hands-on practice
- 24/7 access to learning materials
Learn from anywhere without compromising on quality or interaction.
Self-paced Learning
Our autonomous learning option provides:
- Extensive library of pre-recorded lessons
- Downloadable resources and exercises
- Progress tracking and self-assessment tools
- Community forum for peer support
- Optional mentor access
Ideal for independent learners who need maximum schedule flexibility.
Get Recognized with the Course Completion Certificate
Course completion certificate will be awarded on successful course completion and upon scoring above 65% marks in the quiz consistently throughout the program.
A value add-on to your LinkedIn Profile that can grab attention of our hiring partners and top big data companies.
Student Reviews
Manish offers comprehensive courses that cover Hadoop to spark in a very easy manner. From data ingestion to advanced analytics. His training emphasizes hands-on projects and practical applications for professionals looking to enhance their skills.
Nishant Kumar Saxena
Assistant vice President at Bank of America
Manish Sir has been a very instrumental and thorough professional during our 5-day training on Advance Hadoop. He was extremely patient and helpful during the entirety of the training and was open to take all the doubts of the participants and would ensure everyone would understand the concepts before moving further. I would surely recommend Mr. Manish to all organizations and individuals who would like to gain advance knowledge on a variety of technology skills!! Thank you for everything Sir!! 🙏🏻
Arshan Kika
Sr. Tech Associate at BA Continuum India Pvt. Ltd.
I had attended Manish's training on Big data Hadoop and Spark. It was a nice and detailed training. I like the method, flow , examples and time given to practice all things. He has in depth knowledge of the subject and clarifies all your doubts calmly and completely.I really recommend his training on Big Data Hadoop and Spark.
Nhanu Chavan
Software Engineer III at BA Continuum India Pvt.Ltd
Manish Singh is a seasoned expert in Big Data, Data Science, and AI, with a strong academic and industry background. Here’s a deeper look into his expertise:
🧠 Core Areas of Expertise
-
Big Data Technologies: Skilled in Hadoop, Spark, PySpark, and distributed computing frameworks.
-
Data Science & Machine Learning: Applies statistical modeling, predictive analytics, and deep learning to solve real-world problems.
-
Cloud Computing: Experienced with cloud platforms like AWS and Azure for scalable data solutions.
-
Corporate Training: Known for delivering high-impact training sessions to professionals and organizations on emerging tech topics.
🎓 Academic Foundation
-
Studied at IIT Bombay, one of India’s premier engineering institutes.
-
Focused on databases, data mining, and information retrieval during his academic journey.
- Done Faculty Development Program from IISC , Bangalore, IIT Delhi, IIT Kanpur and Other Top Indian Educational Institute.
🏢 Industry Experience
-
Worked with top-tier companies including Amadeus, Walmart, Ericsson, Oracle, IBM, and Siemens, BOA.
-
Currently Working as a Corporate Trainer at Fortune 500 Companies, helping teams upskill in data-driven technologies.
Python
-
1
Introduction to Python -
2
Data Types
-
3
Statements
-
4
Control structures: if/else, loops, comprehensions
-
5
Functions and scope
-
6
Error handling (try/except)
-
7
Modules and packages
-
8
Virtual environments (venv, pipenv)
-
9
Working with Data
-
10
File handling: read/write to .txt, .csv, .json, .xml
-
11
CSV and JSON parsing
-
12
Working with Excel files (openpyxl, pandas)
-
13
Regular expressions (re module)
-
14
Python Libraries for Data Engineering
-
15
Pandas: data manipulation and transformation
-
16
NumPy: numerical computing
-
17
Databases and SQL with Python
-
18
CRUD operations with SQL
-
19
Query automation and batch execution
-
20
Connecting to databases:
SQL
-
21
SQL Basics
-
22
Data types
-
23
CRUD Operations
-
24
Joins and Set Operations
-
25
INNER JOIN
-
26
LEFT JOIN, RIGHT JOIN
-
27
FULL OUTER JOIN
-
28
Set operations
-
29
UNION vs UNION ALL
-
30
INTERSECT
-
31
EXCEPT / MINUS
-
32
Aggregation and Grouping
-
33
Aggregate functions
-
34
GROUP BY
-
35
HAVING
-
36
nested subqueries
-
37
Common Table Expressions (WITH)
-
38
Window Functions
-
39
ROW_NUMBER(), RANK(), DENSE_RANK()
-
40
LEAD() / LAG()
-
41
PARTITION BY
-
42
ORDER BY
-
43
Aggregates over windows
-
44
Data Modeling and DDL (Data Definition Language)
-
45
Creating databases and tables
-
46
CREATE, ALTER, DROP, TRUNCATE
-
47
Data types, constraints
-
48
PRIMARY KEY
-
49
FOREIGN KEY
-
50
Indexes and Performance Tuning
Dataware House
-
51
Data Warehousing and Analytics SQL
-
52
Star and snowflake schemas
-
53
Slowly Changing Dimensions (SCD Types 1, 2)
-
54
OLAP vs OLTP query patterns
-
55
Time series and cohort analysis in SQL
-
56
Working with Modern SQL Engines
-
57
MySQL
-
58
Microsoft SQL Server
-
59
BigQuery / Snowflake / Redshift / Databricks
-
60
Connecting from Python (psycopg2, sqlalchemy, pandas.read_sql)
-
61
Optional Advanced Topics
Big Data
-
62
Introduction of Big Data
-
63
Distributed System Architecture
-
64
Configuration files
-
65
Cluster Architecture
-
66
Federation and High Availability
-
67
Production Cluster Discussions
-
68
YARN MR Application Execution Flow
-
69
Anatomy of MapReduce Program
-
70
Demo on MapReduce
-
71
Input Splits, Relation between Input Splits and HDFS Blocks
-
72
Map Reduce Algorithm
-
73
Input Splits in MapReduce,Combiner & Partitioner
HIVE
-
74
Hive Introduction
-
75
Hive Introduction
-
76
Hive Architecture and Components
-
77
Metastore in Hive, Limitations of Hive
-
78
Comparison with Traditional Database
-
79
Hive Data Types and Data Models
-
80
Hive Tables
-
81
Managed Tables and External Tables
-
82
Importing Data
-
83
Querying Data
-
84
Partitions and Buckets
-
85
Hive Script
-
86
Hive UDF
-
87
Retail use case in Hive, Hive Demo on Stock Data set
SQOOP
pYSpark
-
92
Spark architecture
-
93
Spark functional operations like map, map partition reduce etc
-
94
What are Dependencies and Why They are Important
-
95
First Program In Spark
-
96
RDDs (Operations, Transformation, Actions)
-
97
Fault Tolerance
-
98
RDD Actions, Data Loading
-
99
Map Reduce - Key value Pair.
-
100
Loading Data in Memory
-
101
Memory Layout In Executor
-
102
Resource Management - Standalone
-
103
Resource Management - Standalone
-
104
Resource Management - YARN
-
105
Dynamic Resource Allocation
-
106
Project on -> STOCK DATA
-
107
What's the Problem with RDDs
-
108
Caching Data In Spark
-
109
DataFrame vs DataSet vs SQL
-
110
Simple Selects
-
111
Filtering DataFrames
-
112
Shuffles and Aggregations DataFrames
-
113
Joining DataFrames
-
114
Learning about Spark SQL
-
115
creating Data Frames, manual inferring of schema
-
116
working with CSV files, reading JDBC tables, Data Frame to JDBC
-
117
SQL Context in Spark for providing structured data processing
-
118
JSON support in Spark SQL
-
119
Types of Joins and optimisations
-
120
Caching and Persistence
-
121
Query and transform data in Data Frames
-
122
Broadcasting variables and accumulators
-
123
Spark resource allocation and configuration
-
124
working with XML data, parquet files
-
125
Creating HiveContext
-
126
Writing Data Frame to Hive
-
127
Shared variable and accumulators
-
128
Spark Streaming Architecture
-
129
Writing streaming program, processing of spark stream
-
130
Request count and Dstream
-
131
Multi batch operation, sliding window operations and advanced data sources
-
132
Different Algorithms, the concept of iterative algorithm in Spark
DataBricks
-
133
Databricks Create A Cluster
-
134
How does Apache Spark run on a cluster?
-
135
Creating Notebooks & few exercise to do in Py-Spark
-
136
How to create and write various scripts and execution
-
137
Create modules and invoke from other notebooks
-
138
Creating Widgets
-
139
File options
-
140
Magic commands
-
141
Databricks specific commands using dbutils library
-
142
Performance Tunning
-
143
DAG Discussion
-
144
Narrow Dependency
-
145
Wide Dependency
-
146
Fail Over
-
147
Cache & persist
-
148
DATA FRAMES
-
149
creating Data Frames, manual inferring of schema
-
150
SQL Context in Spark for providing structured data processing
-
151
JSON support in Spark SQL
-
152
Working with parquet files
-
153
Reading csv files, user defined functions in Spark SQL
-
154
shared variable and accumulators
-
155
learning to query and transform data in Data Frames
-
156
Creating & executing Workflow in Databricks
-
157
Monitoring the Workflow
-
158
Debugging the issues using log in Databricks
-
159
How to create Cluster in Debricks along with do/don’ts, cost involved in utilizing cluster.
-
160
Create cluster / auto scale option / nodes and executors
-
161
Job clusters and all purpose clusters
-
162
Connection pools setup
-
163
Verify driver logs to understand execution of workflow/notebook (DAG)
-
164
Shuffles and Aggregations DataFrames
-
165
Joining DataFrames
-
166
Creating Data Frames, manual inferring of schema
-
167
Distributed Data: The DataFrame
-
168
SQL Context in Spark for providing structured data processing
-
169
JSON support in Spark SQL
-
170
Types of Joins and optimisations
-
171
Caching and Persistence
-
172
Apache Spark Architecture: DataFrame Immutability
-
173
Apache Spark Architecture: Narrow Transformations
-
174
How to Drop rows and columnsworking with XML data, parquet files
-
175
Shared variable and accumulators
-
176
Spark resource allocation and configuration
-
177
Spark performance tuning
-
178
Optimizing spark Jobs, understanding execution plan
-
179
Decide executors and executor cores
-
180
Catalyst optimiser
-
181
Spark Tungsten basics
-
182
Handling Null Values Part II - DataFrameNaFunctions
-
183
Sort and Order Rows - Sort & OrderBy
-
184
Create Group of Rows: GroupBy
-
185
Delta architecture / lakehouse / Medallion architecture
-
186
Fundamentals of Machine Learning with Spark
-
187
Spark Machine Learning components
-
188
Stages in the development of a Machine Learning model
-
189
Machine Learning Model Definition and Pipeline Development
-
190
Model evaluation with PySpark and Databricks
-
191
Spark Optimization Techniques
-
192
Job Stats Discussions
Generative AI
-
196
Introduction to using Generative AI for Data Generation and Augmentation
-
197
Generating Synthetic Data with Generative AI
-
198
Augmenting Existing Data with Generative AI
-
199
Creating Time Series Data
-
200
Generating Edge Cases in Data Engineering
-
201
Handling PII Data with Generative AI
-
202
Balancing Imbalanced Datasets in Data Engineering
-
203
Data Augmentation App Walkthrough
-
204
Creating Functions for Data Engineering
-
205
Running the Web App (GenAI for Data Engineering)
-
206
Introduction to using Generative AI for Writing Data Engineering Code with Gen AI
-
207
Data Cleaning and Modeling with Generative AI
-
208
Documenting Code for Data Projects
-
209
Creating Data Schemas, Systems, and Pipelines
-
210
Transferring Data with Generative AI
-
211
Introduction to using Generative AI for Gen AI Data Engineering Tools
-
212
Use ChatGPT for Data Engineering
-
213
Build a Data Engineering App with Claude
-
214
Custom GPTs for Data Engineering
-
215
Custom LLM or Generative AI tools for Data Engineering
-
216
Copilot for Azure Data Factory and Gemini for BigQuery
-
217
Introduction to using Generative AI for Data Parsing and Extraction
-
218
Parsing Data (Data Engineering)
-
219
Extracting Data from Web Scrapes and Images
-
220
Performing Named Entity Recognition
-
221
Activity: Extracting Data from Contracts
-
222
Introduction to using Generative AI for Data Querying and Analysis
-
223
Querying Data with Generative AI
-
224
Optimizing Data Queries
-
225
Activity: Developing Data Engineering Query Apps
-
226
Running Data Engineering Query Apps
-
227
Converting to a Web App with Front-End
-
228
Generative AI for Data Enrichment
-
229
Enriching Features for Data Models
-
230
Data Imputation and Normalization with Generative AI
-
231
Imputation for Time Series Data Engineering
-
232
Standardizing and Normalizing Textual Data with Generative AI
Who Can Apply for the Data Engineering Course?
•tFreshers and Undergraduates willing to pursue a career in data engineering
•tAnyone looking for a career transition to data engineering
•tIT professionals
•tExperienced professionals willing to learn data engineering
•tTechnical and non-technical professionals with basic-level programming knowledge can also apply
•tProject Managers
•tAnyone looking for a career transition to data engineering
•tIT professionals
•tExperienced professionals willing to learn data engineering
•tTechnical and non-technical professionals with basic-level programming knowledge can also apply
•tProject Managers
What Roles Does a Data Engineer Play?
Big Data Engineer
They design and build complex data pipelines and have expert knowledge in coding using Python, etc. These professionals collaborate and work closely with data scientists to run the code using various tools such as the Hadoop ecosystem, etc.
Data Architect
They are typically the database administrators and are responsible for data management. These professionals have in-depth knowledge of databases, and they also help in business operations.
Business Intelligence Engineer
They are skilled in data warehousing and create dimension models for loading data for large-scale enterprise reporting solutions. These professionals are experts in using ELT tools and SQL.
Data Warehouse Engineer
They are responsible for looking after the ETL processes, performance administration, dimensional design, etc. These professionals take care of the full back-end development and dimensional design of the table structure.
Technical Architect
They design and define the overall structure of a system to improve the business of an organization. The job role of these professionals involves breaking large projects into manageable pieces.
They design and build complex data pipelines and have expert knowledge in coding using Python, etc. These professionals collaborate and work closely with data scientists to run the code using various tools such as the Hadoop ecosystem, etc.
Data Architect
They are typically the database administrators and are responsible for data management. These professionals have in-depth knowledge of databases, and they also help in business operations.
Business Intelligence Engineer
They are skilled in data warehousing and create dimension models for loading data for large-scale enterprise reporting solutions. These professionals are experts in using ELT tools and SQL.
Data Warehouse Engineer
They are responsible for looking after the ETL processes, performance administration, dimensional design, etc. These professionals take care of the full back-end development and dimensional design of the table structure.
Technical Architect
They design and define the overall structure of a system to improve the business of an organization. The job role of these professionals involves breaking large projects into manageable pieces.
•What does a Data Engineer do?
With organizations relying heavily on data to drive growth today, data engineering is becoming a more popular skill. Data engineers are tasked with designing a system to unify multiple sources of business data in a meaningful and accessible way. The typical role of data engineers includes:
•tAcquiring big data sets from different data warehouses
•tCleaning those big data sets and finding any errors
•tRemoving any form of duplications that may occur
•tConverting the cleaned data into a readable format
•tInterpreting data to provide reliable information for better decisions
•tAcquiring big data sets from different data warehouses
•tCleaning those big data sets and finding any errors
•tRemoving any form of duplications that may occur
•tConverting the cleaned data into a readable format
•tInterpreting data to provide reliable information for better decisions
•What are the benefits of taking this Data Engineering course?
This comprehensive data engineering course is designed to provide an introduction to data, a detailed view of the domain and equip you with the skills and techniques to succeed in it. Our course integrates data warehousing, data lakes, and data engineering pipelines to create a comprehensive and scalable data architecture.
•What is the average salary of a Data Engineer?
Today, small and large companies depend on data to help answer important business questions. Data engineering plays a crucial role in supporting this process, making it possible for others to inspect the data available reliably making them important assets to organizations, earning lucrative salaries worldwide. Here are some average yearly estimates:
•tIndia: INR 10.5 Lakhs
•tUS: USD 131,713
•tCanada: CAD 98,699
•tUK: GBP 52,142
•tAustralia:AUD 118,000
•tIndia: INR 10.5 Lakhs
•tUS: USD 131,713
•tCanada: CAD 98,699
•tUK: GBP 52,142
•tAustralia:AUD 118,000
•What will be the career path after completing the Data Engineering Course?
Designing and building data applications is very well regarded in the industry. While becoming a data engineer would be the most obvious route after completing this course, there are several other career paths you could choose:
•tBig Data Engineer: Work with big data technologies like Hadoop and
Kafka to manage data processing tasks.
•tData Architect: Design and manage an organization's data
architecture, ensuring integrity and security.
•tData Analyst: Effectively analyze and interpret complicated data
sets to make informed decisions.
•tBusiness Intelligence Developer: Create and manage BI solutions
using dashboards and reports.
•tBig Data Engineer: Work with big data technologies like Hadoop and
Kafka to manage data processing tasks.
•tData Architect: Design and manage an organization's data
architecture, ensuring integrity and security.
•tData Analyst: Effectively analyze and interpret complicated data
sets to make informed decisions.
•tBusiness Intelligence Developer: Create and manage BI solutions
using dashboards and reports.
•Do Data Engineers require prior coding experience?
Yes, data engineers are expected to have basic programming skills in Java, Python, R or any other language.
•Can I apply for this data engineering course with no technical background?
Yes, you can join this data engineering course even with no technical background. However, it's recommended that you have a basic understanding of object-oriented programming languages and at least two years of relevant work experience.
•Which are the top industries suitable for Data Engineering professionals?
Organizations around the world are looking for ways to leverage data to enhance services, making data engineers a sought-after asset. That being said, some of the more popular industries for data engineers include:
•tMedicine and healthcare
•tBanking
•tInformation technology
•tEducation
•tRetail
•tEcommerce
•tMedicine and healthcare
•tBanking
•tInformation technology
•tEducation
•tRetail
•tEcommerce
•What is the refund policy for this Data Engineering Course?
submit your refund request within 7 days of purchasing your course, 100 % Money will be refunded ( No Question asked )
•After 7 days -No Refund are Allowed in any Circumstances.
•After 7 days -No Refund are Allowed in any Circumstances.
•Are there any other online courses Times Analytics offers under Data Science?
Absolutely! Times Analytics offers plenty of options to help you upskill in Data Science. You can take advanced certification training courses or niche courses to sharpen specific skills. Whether you want to master new tools or stay ahead with the latest trends, there's something for everyone. These courses are designed to elevate your knowledge and keep you competitive in the Data Science field.
Similar programs that we offer under Data Science:
Similar programs that we offer under Data Science:
•Will missing a live class affect my ability to complete the course?
No, missing a live class will not affect your ability to complete the course. With our 'flexi-learn' feature, you can watch the recorded session of any missed class at your convenience. This allows you to stay up-to-date with the course content and meet the necessary requirements to progress and earn your certificate.
•Does Times Analytics have corporate training solutions?
Yes, Times Analytics for Business offers learning solutions for the latest AI and other digital skills, including industry certifications. For talent development strategy, we work with Fortune 500 and mid-sized companies with short skill-based certification training and role-based learning paths.
Our team of curriculum consultants works with each client to select and deploy the learning solutions that best meet their teams’ requirements.
Our team of curriculum consultants works with each client to select and deploy the learning solutions that best meet their teams’ requirements.
What can I expect from these courses offered by Times Analytics?
Times Analytics’s online Data Engineering courses will validate your skills in the domain and will add value to your resume. The real-life practical applications will help you develop a strong skill set that you can showcase to recruiters. Get the best Data Engineer course certification and excel in your data engineering career.
How will I get certified?
On the completion of the online data engineering online course, and the completion of the various projects and assignments in this program, you will receive your data engineering certification
Which are the top companies hiring in this domain?
The top companies hiring data engineers around the globe are as follows:
•tTata Consultancy Services (TCS)
•tLTI
•tAccenture
•tAmazon
•tInfosys
•tCapgemini
•tTata Consultancy Services (TCS)
•tLTI
•tAccenture
•tAmazon
•tInfosys
•tCapgemini
I don't have any technical background. Can I still enroll in this Data engineer certification course?
Yes, you can easily join the data engineering courses even if you do not have technical experience or are not from a technical background. However, knowing any object-oriented programming language will be helpful.
How much time should I devote to completing this program?
The data engineering courses come with a duration of 2 months of live classes and lifelong access to course material. In this tenure, it is suggested that you devote six to seven hours a week to master the data engineering concepts taught in the online classes.
