We were recently working on a customer project where we were leveraging AWS Lambdas to build a REST API backed by AWS API Gateway. Re-usability of your code in serverless environments is not as straightforward as is with its non-serverless counterparts so depending on what your functions are doing, you could find yourself with a good amount of duplicated code across your different functions. To address this limitation, AWS added support for a feature called Lambda Layers. Unfortunately, Layers can create some nontrivial problems for unit testing your code in your local environment. This post will walk through an approach to making local testing possible.

What are Lambda Layers?

AWS Lambda Layers provide a good avenue for reusing code across your different lambda functions and keeping your deployment packages small. The explanation from AWS reads as follows:

“Lambda Layers is a way to centrally manage code and data that is shared across multiple functions.”

 

This post isn’t a tutorial on Layers and expects some basic Layers knowledge. If you want a quick tutorial on how to get started with Layers, I recommend this read by Adrian Hornsby.

The Challenge with Layers

One of the really nice benefits of using Layers is that you can create reusable modules and not have to worry about how to publish them as artifacts for each of your different functions to consume. The problem though is that the module resolution is provided by the AWS Lambda runtime which means that if you are trying to reference your layer code locally, the IDE interpreter won’t be able to find it and it will fail during local execution. Alternatively, you could use AWS SAM but now you have additional dependencies to manage in your environment which makes portability that much harder. This creates a bit of a problem if you’re trying to unit-test your lambda, which for us is a must-have!

What Can We Do?

To resolve this, you can use a factory method pattern to encapsulate an accessor for the layer’s code and use conditional imports to load your code as necessary. Then, during local execution and unit testing you can mock out the code provided by the layer, and at runtime, the layer can be loaded without trouble as long as you configure your lambda layer correctly.

Let’s see it in action!

Let’s assume we create a Python module responsible for authorization that we want to reuse across our code: the authorizer class has 3 static methods that are useful in many of our lambda functions. 2 of the methods defer authorization to another internal class of the module for illustration purposes while the 3rd method is used to construct an object that is used by the authorizer.

from authorization_request import AuthorizationRequestfrom 
authorization_response import AuthorizationResponsefrom 
repository_factory import REPOSITORY_FACTORYclass 
Authorizer:@staticmethoddef 
retrieve_authorized_records_by_roles(user_roles):repo = 
REPOSITORY_FACTORY.get_repository()return 
repo.get_records_mapped_to_roles(user_roles)@staticmethoddef 
is_authorized(authorization_request: AuthorizationRequest):repo = 
REPOSITORY_FACTORY.get_repository()return 
repo.is_authorized(authorization_request)@staticmethoddef 
create_authorization_request(roles, record_ids):request = AuthorizationRequest()request.user_roles = rolesrequest.record_ids = record_idsreturn request

In our Lambda function we use a factory pattern as well to load our authorizer class like this:

authorizer = 
FACTORY.get_authorizer()authorization_request 
=authorizer.create_authorization_request(roles, record_ids)response = 
authorizer.is_authorized(authorization_request)if not 
response.get('fully_authorized'):raise UnauthorizedRequestError(f"Not 
authorized to retrieve records {response.get('unauthorized_records')}")

As can be seen in the above snippet from our lambda function, we use a FACTORY class to retrieve the authorizer, then we construct an authorization_request using the authorizer and finally call the is_authorized method from the authorizer. If the request is not fully_authorized, we raise an exception with the unauthorized records. This pattern is helpful because it allows us to encapsulate the loading logic behind the factory and in using Factory initialization for unit test purposes.

Let’s take a look at the factory now:

class Factory:def __init__(self):self._authorizer = Nonedef 
set_authorizer(self, authorizer):self._authorizer = authorizerdef 
get_authorizer(self):if not self._authorizer:# noinspection 
PyUnresolvedReferencesfrom authorizer import Authorizer  # pylint: 
disable=E0401self._authorizer = Authorizer()return 
self._authorizerFACTORY = Factory()

This simple class allows us to set an authorizer and retrieve an authorizer. The reason we have a setter is so that we can set a mock authorizer for the purpose of unit testing.The magic happens in the getter. Notice how we first check if an authorizer has not been set. In that case, we use a conditional import to instantiate the Authorizer that will be provided from the module injected by the Layer. We set the internal authorizer of the factory and return it.This pattern allows us to swap the Authorizer implementation for the purpose of unit testing while loading the real authorizer at runtime.

# noinspection PyUnresolvedReferencesfrom authorizer import Authorizer  
# pylint: disable=E0401

Notice the comments surrounding the conditional import. These are here because the IDE (In our case IntelliJ) correctly identifies that it can’t find the authorizer module. Similarly we are using pylint for code linting which also identifies that it can’t find the module and without the comments we have it configured to fail the build. The above comments tell both the IDE and pylint to ignore the module that it can’t resolve since we know this will be resolvable in AWS thanks to the Lambda runtime and the proper configuration of the Authorizer Layer.

How do the unit tests look?

import unittestfrom unittest.mock import Mock, MagicMockimport 
lambda_functionfrom factory import FACTORYclass 
TestLambdaFunction(unittest.TestCase):def setUp(self):mock_authorizer = 
Mock()mock_authorizer.is_authorized =MagicMock(return_value=
{'fully_authorized': True})FACTORY.set_authorizer(mock_authorizer)def 
test_request_given_unauthorized_ids_should_return_403(self):mock_author
izer = Mock()mock_authorizer.is_authorized = MagicMock(return_value=
{'fully_authorized': False,'unauthorized_records': [1, 2, 
3]})FACTORY.set_authorizer(mock_authorizer)# … remainder of the test

The above block shows that each test setup creates a mock Authorizer and uses the static FACTORY class to set it. If you recall, the same FACTORY class is used by the lambda function to load the authorizer which in the case of our unit tests will be getting the injected Mock.You can also see from the block above how we can create a different mock in a particular test to make sure the function responds accordingly when a nonauthorized request is made!

In Summary:

Layers can be quite useful to share and reuse code across all of your Lambda functions but require a bit of thought and proper organization of your code to encapsulate things in a way that makes it easily testable. Thanks to the Factory pattern and Python’s conditional imports we can get around the limitation and still unit test our code without having to install other dependencies like AWS SAM.

Let me know what you think and how you have approached this problem in the past. Are you using Lambda Layers in your projects? If not, why not? If so, how have you been unit-testing your code that depends on the Layers? Please let me know through the comments, I would love to know what others are doing!