AWS Lambda provides an amazing feature called the layer which allows you to add your custom library. It allows you to stop code repetition. In this article, we are going to see how to achieve this.
In case you are someone new to AWS then take a look at this article to know what AWS is and what solution they offer. Also, look into the lambda introduction if you are not aware of AWS lambda.
- Preparing the python package
- Creating lambda layer
- Adding layer to lambda
- Common issues
- useful resources and tips
Preparing the python package
One thing which you must keep in mind when importing a python package to lambda is that the root folder must be exactly named as “python”.
Create a folder called python and open the command prompt (pointing to the python directory). You can add your custom scripts to it. If you want to install an existing python library do the following.
Install the package which you wish to import using the following command.
pip install {YOUR_PACKAGE_NAME} -t .
After the library is downloaded, zip the “python” folder.
Creating lambda layer
Open the AWS lambda console and click on “layers” which is shown at the left.
Click create “layer”, upload the zip file and select the runtime programming languages (you can also select multiple versions of python).
Adding layers to lambda
Go to your python lambda function and scroll down to the bottom. You should find “add layer”.
Click it and add the layer you imported to lambda.
Common issues
- If you could not find your imported layer in your lambda, then most likely you did not specify the appropriate python version when selecting the runtime during layer import. You can add another version of the same layer and select the appropriate version to solve this issue.
- If you get the error “module not found”, then you need to check whether the root directory of your layer is named “python”.
- Another most common issue would be that the python version of your local machine and AWS lambda would not match and consequently you may face an issue. To resolve it, you can follow a method that I hate, which is to SSH to an Amazon Linux machine and repeat the same pip install commands and upload to S3. Download the zip file from S3 and upload to the layer. If you find it as too much work like me, then continue reading.
Useful resources and tips
When you want to add a python library of the previous version then the easiest way would be to google for the lambda layer on the internet. I personally update the lambda whichever I use to my public Github repository. I hope you find the lambda you are looking for in this repository.
https://github.com/ranjithkumarmadhavan/lambda-python-layers
If that does not work, you could try to search for a “.whl” file of that library. A “.whl” file can be installed using the following commands.
pip install {YOUR_WHEEL_FILE.whl} -t .
One of the websites which I personally use to download these wheels files is https://www.lfd.uci.edu/~gohlke/pythonlibs/.
Happy Programming!!!
1 thought on “Add layers, troubleshoot and common mistakes in AWS lambda”