Docker Image for AWS Lambda Succeeds in CodeBuild but Fails to Run Correctly

Archiee · August 27, 2024, 10:32am

I’m trying to build a Docker image for an AWS Lambda function which is a PHP API, using a custom Dockerfile. I recently updated the Dockerfile to use Amazon Linux 2 (amazon/aws-lambda-provided:al2) as the base image, but while the CodeBuild succeeds, the resulting Docker image seems to be incompatible with the application code when deployed to AWS Lambda.

Old Dockerfile (Works in Lambda but fails CodeBuild): The Lambda image that this was using from 2 years ago is working, and the API functionality was undisturbed in the application. But when I merge the code and the codebuild was building I got a failure in build with an error like this.

Status: Downloaded newer image for public.ecr.aws/lambda/provided:latest
 ---> xxxxxxxxxxxxx
Step 2/17 : ARG php_version="8.0.16"
 ---> Running in bc5c374a2d1b
Removing intermediate container xxxxxxxxxxxxx
 ---> xxxxxxxxxxxxx
Step 3/17 : RUN yum clean all &&     yum install -y autoconf                 bison                 bzip2-devel                 gcc                 gcc-c++                 git                 gzip                 libcurl-devel                 libxml2-devel                 make                 openssl-devel                 tar                 unzip                 zip                 openldap-devel                 libzip010-compat                 re2c                 sqlite-devel                 oniguruma                 oniguruma-devel                 file                 wget                 libldb-devel
 ---> Running in xxxxxxxxxxxxx
/bin/sh: line 1: yum: command not found
The command '/bin/sh -c yum clean all &&     yum install -y autoconf                 bison                 bzip2-devel                 gcc                 gcc-c++                 git                 gzip                 libcurl-devel                 libxml2-devel                 make                 openssl-devel                 tar                 unzip                 zip                 openldap-devel                 libzip010-compat                 re2c                 sqlite-devel                 oniguruma                 oniguruma-devel                 file                 wget                 libldb-devel' returned a non-zero code: 127

[Container] 2024/08/14 17:29:58.114652 Command did not exit successfully docker build -t $REPOSITORY_URI:latest . exit status 127
[Container] 2024/08/14 17:29:58.119545 Phase complete: BUILD State: FAILED
[Container] 2024/08/14 17:29:58.119564 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker build -t $REPOSITORY_URI:latest .. Reason: exit status 127
[Container] 2024/08/14 17:29:58.158806 Entering phase POST_BUILD
[Container] 2024/08/14 17:29:58.159700 Running command echo Build completed on `date`
Build completed on Wed Aug 14 17:29:58 UTC 2024

[Container] 2024/08/14 17:29:58.167271 Running command echo Pushing the Docker images...
Pushing the Docker images...

[Container] 2024/08/14 17:29:58.173839 Running command docker push $REPOSITORY_URI:latest
The push refers to repository [xxxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/php-lambda]
An image does not exist locally with the tag: xxxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/php-lambda

[Container] 2024/08/14 17:29:58.196466 Command did not exit successfully docker push $REPOSITORY_URI:latest exit status 1
[Container] 2024/08/14 17:29:58.201397 Phase complete: POST_BUILD State: FAILED
[Container] 2024/08/14 17:29:58.201417 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker push $REPOSITORY_URI:latest. Reason: exit status 1
[Container] 2024/08/14 17:29:58.249533 Set report auto-discover timeout to 5 seconds
[Container] 2024/08/14 17:29:58.249589 Expanding base directory path:  .

Original Docker File:

# Lambda base image Amazon Linux
FROM public.ecr.aws/lambda/provided as builder 

# Set desired PHP Version
ARG php_version="8.0.16"

RUN yum clean all && \
    yum install -y autoconf \
                bison \
                bzip2-devel \
                gcc \
                gcc-c++ \
                git \
                gzip \
                libcurl-devel \
                libxml2-devel \
                make \
                openssl-devel \
                tar \
                unzip \
                zip \
                openldap-devel \
                libzip010-compat \
                re2c \
                sqlite-devel \
                oniguruma \
                oniguruma-devel \
                file \
                wget \
                libldb-devel

# Add libsodium needed for PHP 8
RUN wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm && \
    rpm -Uvh epel-release-latest-7.noarch.rpm && \
    yum install -y libsodium \
                   libsodium-devel

# Download the PHP source, compile, and install both PHP and Composer
RUN curl -sL https://github.com/php/php-src/archive/php-${php_version}.tar.gz | tar -xvz && \
    cd php-src-php-${php_version} && \
    ./buildconf --force && \
    ./configure --prefix=/opt/php-8-bin/ --with-openssl --with-curl --with-zlib --without-pear --enable-bcmath --with-bz2 --enable-mbstring --with-libdir=lib64 --with-ldap --with-mysqli && \
    make -j 5 && \
    make install && \
    /opt/php-8-bin/bin/php -v && \
    curl -sS https://getcomposer.org/installer | /opt/php-8-bin/bin/php -- --install-dir=/opt/php-8-bin/bin/ --filename=composer

# Prepare runtime files
COPY runtime/bootstrap /lambda-php-runtime/
RUN chmod 0755 /lambda-php-runtime/bootstrap

# Install Guzzle, AWS SDK, and prepare vendor files
RUN mkdir /lambda-php-vendor && \
    cd /lambda-php-vendor && \
    /opt/php-8-bin/bin/php /opt/php-8-bin/bin/composer require guzzlehttp/guzzle && \
    /opt/php-8-bin/bin/php /opt/php-8-bin/bin/composer require aws/aws-sdk-php

# Clean up
RUN yum clean all && \
    rm -rf /var/cache/yum

###### Create runtime image ######
FROM public.ecr.aws/lambda/provided as runtime

# Layer 1: PHP Binaries
COPY --from=builder /opt/php-8-bin /var/lang

# Layer 2: Runtime Interface Client
COPY --from=builder /lambda-php-runtime /var/runtime

# Layer 3: Vendor
COPY --from=builder /lambda-php-vendor/vendor /opt/vendor

COPY src/ /var/task/

# Install PHP 8 libs
RUN yum install -y oniguruma \
    oniguruma-devel

# Clean up
RUN yum clean all && \
    rm -rf /var/cache/yum

CMD [ "index" ]

** New Dockerfile (Succeeds in CodeBuild but fails in Lambda):** This Dockerfile Succeeded in the Codebuild and performed without any errors, but when I deploy the image created by this build in the Lambda it is disrupting the API functionality that my application requires. Here’s the Codebuild log where I think there might be an issue.

Successful Codebuild log where issue might be stemming from.

Codebuild status:

# Lambda base image Amazon Linux 2
FROM amazon/aws-lambda-provided:al2 as builder

# Set desired PHP Version
ARG php_version="8.0.16"

RUN yum clean all && \
    yum install -y autoconf \
                bison \
                bzip2-devel \
                gcc \
                gcc-c++ \
                git \
                gzip \
                libcurl-devel \
                libxml2-devel \
                make \
                openssl-devel \
                tar \
                unzip \
                zip \
                openldap-devel \
                libzip010-compat \
                re2c \
                sqlite-devel \
                oniguruma \
                oniguruma-devel \
                file \
                wget \
                libldb-devel \
                libsodium \
                libsodium-devel

# Download the PHP source, compile, and install both PHP and Composer
RUN curl -sL https://github.com/php/php-src/archive/php-${php_version}.tar.gz | tar -xvz && \
    cd php-src-php-${php_version} && \
    ./buildconf --force && \
    ./configure --prefix=/opt/php-8-bin/ --with-openssl --with-curl --with-zlib --without-pear --enable-bcmath --with-bz2 --enable-mbstring --with-libdir=lib64 --with-ldap --with-mysqli && \
    make -j 5 && \
    make install && \
    /opt/php-8-bin/bin/php -v && \
    curl -sS https://getcomposer.org/installer | /opt/php-8-bin/bin/php -- --install-dir=/opt/php-8-bin/bin/ --filename=composer

# Prepare runtime files
COPY runtime/bootstrap /lambda-php-runtime/
RUN chmod 0755 /lambda-php-runtime/bootstrap

# Install Guzzle, AWS SDK, and prepare vendor files
RUN mkdir /lambda-php-vendor && \
    cd /lambda-php-vendor && \
    /opt/php-8-bin/bin/php /opt/php-8-bin/bin/composer require guzzlehttp/guzzle && \
    /opt/php-8-bin/bin/php /opt/php-8-bin/bin/composer require aws/aws-sdk-php

# Clean up
RUN yum clean all && \
    rm -rf /var/cache/yum

###### Create runtime image ######
FROM public.ecr.aws/lambda/provided as runtime

# Layer 1: PHP Binaries
COPY --from=builder /opt/php-8-bin /var/lang

# Layer 2: Runtime Interface Client
COPY --from=builder /lambda-php-runtime /var/runtime

# Layer 3: Vendor
COPY --from=builder /lambda-php-vendor/vendor /opt/vendor

COPY src/ /var/task/

CMD [ "index" ]

Question How can I resolve the compatibility issue with my new Docker image so that it runs correctly on AWS Lambda? Are there any known issues or differences with Amazon Linux 2 that I should account for when building the image?

Any guidance on identifying the specific incompatibilities or suggestions for modifying my Dockerfile would be greatly appreciated! I’m new to AWS and Docker so it’s a steep learning for me, please be descriptive if y’all have a solution Jr. Dev here!! :')

Verified that all dependencies required for PHP 8.0.16 are installed. Checked that the bootstrap file and other runtime files have the correct permissions. Made sure that all necessary PHP extensions are compiled and installed. Ensured that the Lambda function handler matches the CMD specified in the Dockerfile.

Neeraj · August 28, 2024, 9:19am

Use dnf for Package Management:

Replace all instances of yum with dnf in your Dockerfile’s RUN commands. Here’s the updated RUN command:

Dockerfile

RUN dnf clean all && \
    dnf install -y autoconf \
                bison \
                bzip2-devel \
                gcc \
                gcc-c++ \
                git \
                gzip \
                libcurl-devel \
                libxml2-devel \
                make \
                openssl-devel \
                tar \
                unzip \
                zip \
                openldap-devel \
                libzip010-compat \
                re2c \
                sqlite-devel \
                oniguruma \
                oniguruma-devel \
                file \
                wget \
                libldb-devel \
                libsodium \
                libsodium-devel

Install Necessary Dependencies:

Carefully review the compatibility documentation for your PHP version and the AWS Lambda environment.
Identify any missing dependencies specific to AL2 and add them to your dnf command. You’ve already verified libsodium and libsodium-devel , but there might be others.

Address PHP Configuration Issues:

If specific PHP configuration options are required for your application, you can modify the php.ini file during the build process. Consider using a configuration management tool like puppet or ansible to ensure consistency across environments.

Additional Considerations:

Codebuild and Lambda Environment Differences: While Codebuild might successfully build the image, Lambda might have additional restrictions or limitations. Ensure that your code and dependencies are compatible with both environments.
Incremental Testing: Test your application thoroughly at each stage of development and during the deployment process to identify compatibility issues early on.

Steps to Identify Incompatibilities:

Compare Package Lists: Examine the installed packages in both the old and new images. You can use tools like rpm -qa to list package names. Look for discrepancies and investigate missing dependencies in the AL2 image.
Check PHP Configuration: Compare the php.ini files from both images. Ensure all required extensions and settings are present in the AL2 image.
Run the Application in a Docker Container: Create a Docker container based on the new image, copy your application code, and try to run it. This might reveal functional issues.

By following these steps and carefully considering the potential causes, you should be able to resolve the compatibility issue and successfully deploy your PHP API as a Lambda function using the Amazon Linux 2 base image.