Month: January 2022

Configure Delta Lake – Data Sources and Ingestion

Configure Delta Lake – Data Sources and Ingestion

df = spark.read \.option(“header”,”true”).csv(“/FileStoretablesbrainwavesMeditation.csv”)df.write.mode(“overwrite”).format(“delta”).save(“/FileStore/   data/2022/03/14”)brainwaves=spark.read.format(“delta”).load(“FileStoredata/2022/03/14”)display(brainwaves)print(brainwaves.count()) 5. Run the following code snippet: display(spark.sql(“DROP TABLE IF EXISTS BRAINWAVES”))display(spark \.sql(“CREATE TABLE BRAINWAVES USING DELTALOCATION’FileStoredata/2022/03/14′”))display(spark.table(“BRAINWAVES”).select(“*”).show(10))print(spark.table(“BRAINWAVES”).select(“*”).count())6. Upload the brainwavesPlayingGuitar.csv file using the same process performed in step 2 ➢ navigate…