Roles and Responsibilities
- Source code review, debugging and optimization.
- Investigating performance lagging factors, used python profiler to optimize functions and API response time to improve the visualization projection.
- Experimenting with different data models (netCDF and Zarr) for big data to get better data access response time in the cloud environment.
- Implementing an automation pipeline to convert periodic data from netCDF to Zarr.
- Cloud designing and implementation with Docker and implementing deployment automation with Terraform on the OpenStack.
- Planning and testing a cloud migration for an on-premise application to cloud in docker container environment.
Project information
- Category: Cloud and Development
- Client: Department of Fisheries and Ocean Canada
- Project date: April, 2020 - December, 2020
- Project URL: Source Code
Project Description
Online data visualization solution for ocean data. I worked on a cloud solution design with Docker and Terraform. Used different data models (netCDF and Zarr with Dask) to improve the data access latency. Worked on python Flask backend to improve the API response time.
| Python | API | Docker | Terraform | Openstack | netCDF | Zarr |