In this first exercise you will practice your Python skills,
as well as learn about a very common task ... downloading data files
from a HTTP source.
You will have to unzip the files with Python as well.
-
Change directories at the command line to be inside the
Exercise-1foldercd Exercises/Exercise-1 -
Run
docker build --tag=exercise-1 .to build theDockerimage. -
There is a file called
main.pyin theExercise-1directory, this is where youPythoncode to complete the exercise should go. -
Once you have finished the project or want to test run your code, run the following command
docker-compose up runfrom inside theExercises/Exercise-1directory
You need to download 10 files that are sitting at the following specified
HTTP urls. You will use the Python package requests to do this
work.
You will need to pull the filename from the download uri.
The files are zip files that will also need to be unzipped into
their csv format.
They should be downloaded into a folder called downloads which
does not exist currently inside the Exercise-1 folder. You should
use Python to create the directory, do not do it manually.
Generally, your script should do the following ...
-
create the directory
downloadsif it doesn't exist -
download the files one by one.
-
split out the filename from the uri, so the file keeps its original filename.
-
Each file is a
zip, extract thecsvfrom thezipand delete thezipfile. -
For extra credit, download the files in an
asyncmanner using thePythonpackageaiohttp. Also try usingThreadPoolExecutorinPythonto download the files. Also write unit tests to improve your skills.
- Don't assume all the uri's are valid.
- One approach would be the
Pythonmethodsplit()to retrieve filename for uri, or maybe find the last occurrence of/and take the rest of the string.