staging_events: event data telling what users have done (columns: event_id, artist, auth, firstName, gender, itemInSession, lastName, length, level, location, method, page, registration, sessionId, song, status, ts, userAgent, userId).Location: US-West-2 (as Project-3's AWS S3 bucket).Used set-up in the Project-3 is as follows: AWS Redshift set-upДЄWS Redshift is used in ETL pipeline as the DB solution. The Fact Table answers one of the key questions: what songs users are listening to. Start design means that it has one Fact Table having business data, and supporting Dimension Tables. Sparkify analytics database (called here sparkifydb) schema has a star design. As technologies, Project-3 uses python, SQL, AWS S3 and AWS Redshift DB. Project builds an ETL pipeline (Extract, Transform, Load) to create the DB and tables in AWS Redshift cluster, fetch data from JSON files stored in AWS S3, process the data, and insert the data to AWS Redshift DB. s3://udacity-dend/log_data: 31 files, 8056 DB rows.Д«elow, some figures about the data set (results after running the etl.py): who listened what song, when, where, and with which client S3://udacity-dend/log_data: event data of service usage e.g.

S3://udacity-dend/song_data: static data about artists and songs Data set is a set of files in JSON format stored in AWS S3 buckets and contains two parts: This Project-3 handles data of a music streaming startup, Sparkify. python3 etl.py (to process all the input data to the DB).python3 create_tables.py (to create the DB to AWS Redshift).Copy log_data and song_data folders to your own S3 bucket.ДЄfter installing python3 + AWS SDK (boto3) libraries and dependencies, run from command line:.Edit cwh.cfg: add your S3 bucket name in LOG_PATH and SONG_PATH variables.To run the script to use that data, do the wfollowing: dwhCluster => HOST)ДЇor creating IAM role, getting ARN and running cluster, you can use Udacity-DEND-Project-3-AWS-Setup.ipynb.Д®xample data is in data folder. dwhRole) with AmazonS3ReadOnlyAccess access rights To access AWS, you need to do in AWS the following: Fill in AWS acces key (KEY) and secret (SECRET). PROJECT-3: Data Warehouse Quick startДЇirst, rename dwh_template.cfg to dwh.cfg and fill in the open fields. (Udacity: Data Engineering Nano Degree) | | This project is a part of Udacity's Data Engineer Nano Degree.

Project aho not starting