Big Data Processing using Apache Spark and Amazon Web Services

Jul 24, 2021

--

Big Data Analytics

Aditya Kunar

Written by Aditya Kunar

I am a researcher at Generatrix- An AI-based privacy preserving data synthesizing platform. I have an avid passion for new and emerging technologies in AI & ML

More from Aditya Kunar

Object Detection with SSD and MobileNet

Aditya Kunar

Object Detection with SSD and MobileNet

1. Introduction

24 min readJul 6, 2020

--

1

CTAB-GAN: Effective Table Data Synthesizing

Aditya Kunar

CTAB-GAN: Effective Table Data Synthesizing

By Zilong Zhao, Aditya Kunar, Hiek Van der Scheer, Robert Birke & Lydia Y. Chen

5 min readJun 7, 2021

--

1

DTGAN: Differential Private Training for Tabular GANs

Aditya Kunar

DTGAN: Differential Private Training for Tabular GANs

By Aditya Kunar, Robert Birke, Zilong Zhao & Lydia Y. Chen

6 min readAug 2, 2021

--

See all from Aditya Kunar

Recommended from Medium

Big Data ETL with AWS Glue

AmritKmr

Big Data ETL with AWS Glue

When it comes to loading Big Data from an S3 bucket to a DynamoDB Table, conventional lambda and/or python scripts will fail to deliver…

10 min readAug 30

--

Seamless Data Processing: Spark Structured Streaming for AWS Kinesis Message Streams

Gaurav Patil
in
Globant

Seamless Data Processing: Spark Structured Streaming for AWS Kinesis Message Streams

Harnessing Spark Structured Streaming for efficient AWS Kinesis message processing and analysis

9 min readJun 22

--

Lists

New_Reading_List

174 stories209 saves

Natural Language Processing

924 stories438 saves

Change Data Capture (CDC) Realtime Streaming with Postgres, Debezium, Kafka, Apache Spark, and…

Yusuf Ganiyu
in
Python in Plain English

Change Data Capture (CDC) Realtime Streaming with Postgres, Debezium, Kafka, Apache Spark, and…

In the dynamic world of data processing and analytics, Change Data Capture (CDC) stands out as a critical technology for real-time data…

8 min read6 days ago

--

Data Engineering End-to-End Project — Part 1 — Spark, Kafka, Elasticsearch, Kibana, MinIO, Docker…

Dogukan Ulu

Data Engineering End-to-End Project — Part 1 — Spark, Kafka, Elasticsearch, Kibana, MinIO, Docker…

Repository

8 min read5 days ago

--

1

Small File, Large Impact — Addressing the Small File Issue in Spark

Siraj

Small File, Large Impact — Addressing the Small File Issue in Spark

Introduction:

4 min readJul 14

--

PySpark for AWS Glue: A Comprehensive Guide to Big Data Processing

Shuvradeep Ghosh

PySpark for AWS Glue: A Comprehensive Guide to Big Data Processing

Introduction

10 min readAug 6

--

See more recommendations

Help
Status
About
Careers
Blog
Privacy
Terms
Text to speech
Teams