Add files via upload
Created by: jingrugong
i haven't work on the scripts after cmd 8, so ignore those after i connect with the aws bucket, i change the data path from "C:/Users/Sagar/Documents/TRCdata/Building_audit_data/Data_trc" to "dbfs:/mnt/trc.blocpower.io" as you can see in cmd 6: determine the data path but after running the cmd 7, the resulting address is null, and i tried to store the data locally, and the result is null as well, so i think there is something wrong with the original code my question is: i don't really know where the bug is, since theres no error, and i am sure my path is right and everything is connected my possible solutions are:
- i know there are ways of reading excel files in spark, but it is more complicated and since i am not really familiar with excel, it might take me some time to resolve it (i tried to import com.crealytics:spark-excel_2.11:0.12.2 libraries, but it failed)
- i can convert all data into csv, with different tabs into different csv files, then rewrite it
if you can have any suggestions on where this is wrong, i think it would be much faster than i research it alone. thanks!