CodeBuild is that you just take a source of code, For example, CodeCommit, Amazon S3, Bitbucket or GitHub. And then in that source, there will be some build instructions.
Now, from an exam perspective, you need to know the name of that file. So it’s buildspec.yml, and that file needs to live at the root of your code. And I will show this, what it means, in the hands-on, or you can also insert these instructions already in the console, but the best practice is to use, buildspec.yml. And this is what the exam will test you on.
So once the application is built, the output logs can be stored into Amazon S3 and CloudWatch logs for later analysis, you use CloudWatch Metrics to look at the builds statistics, EventBridge to detect failed builds and trigger notifications, CloudWatch alarms in case you have too many failures, for example, and then the build projects themselves can be, and this is a bit confusing, defined either within CodeBuild of course, or also within CodePipeline, but CodePipeline could also invoke an existing CodeBuild build project.
Supported Environments
So, if we look at CodeBuild, what can it test?
Well, if you have a Java, Ruby, Python, Go, Node.js, Android, .NET Core, PHP application, Then there is a prebuilt image for you to run your test in CodeBuild. And if you want to have any other environments, you can extend a Docker image, and by extending it, you can test for whatever language you want.
But again, this is up to you, to support your own environments.
How it Works
So how does CodeBuild works?
Well, we have our code, and in this example, is in CodeCommits. So we have our source code, a bunch of files, and then there’s this very important file that lives at the top of your repo, which is buildspec.yaml. Now CodeBuild is going to fetch this code and then, CodeBuild itself, will have to have a container.
So, as I said, there’s going to be a build environment. So Java, Go and so on.
And this container is going to load all the source codes and the buildspec.yml and is going to run all the instructions that are inserted into this buildspec.yml file. Now to build this container CodeBuild will pull a Docker image. So either it’s pre-packaged by AWS for the environments I just told you before, or you provide your own Docker image to run whatever code you need.
Okay, so CodeBuild will run all the instructions from buildspec.yml, and sometimes they can be quite lengthy. So there is a feature and CodeBuild to be able to cache a bunch of files in S3 buckets. If you want you to reuse some files from build to build, okay, this is an optimization, but there’s a way for you to cache some files.
This is optional though. Then all the logs are going to be into CloudWatch logs and Amazon S3 if you enable it, and then once CodeBuild is done, to build your code or even test your code, it can produce some artifacts, and these artifacts will be extracted out of the container, put into an S3 bucket, and this is where you can find your final outputs of CodeBuild.
So the buildspec.yml file is super important.
buildspec.yml
This is what it looks like, we’ll have a look at it, obviously in this course. But some very important things are that the buildspec.yml file must at the roots of your code. So at the very, very top of your code directory.
Environment allows you to define some environmental roles for the execution of buildspec.yml.
So you have variables that can be plain text or you can pull them directly from the SSM Parameter Store, or you can pull secrets directly from Secrets Manager. This would allow you, for example, to get a password or for database, and so on directly from some places, that of course you wouldn’t want to store these passwords in plain text into a file like buildspec.yml.
Then phases, which is actually going to define what CodeBuild is going to be doing, so it’s a bunch of installs, for example, to say, “Hey, what commits you wanna do”, to install some pre necessary packages and so on.
Pre-build, which is the commands to execute. just before the build. Build, the actual build commands, very important. Then we have post_build, which are the finishing touches.
For example, once it’s built, maybe create a good zip output and so on, and then the artifacts. So which files at the Docker container should be extracted and sent into Amazon S3. That should also going to be encrypted.
And finally, there’s this cache block, to say which files your dependencies are going to be cached in Amazon S3 for speeding up the future builds.
Okay, so this is, could be at a high level, but from this file, just remember, to me, the most important thing is the name of the file, where it sits. And again, just understand the general idea of how CodeBuild works and you should be good to go.
Local Build
Now CodeBuild is something that runs on the cloud, but it is possible for you, if you need to do some deep troubleshooting beyond the logs to run CodeBuild locally on your desktop.
First you need to install Docker, obviously, and then you leverage the CodeBuild Agent, the instructions are here, and this allows you to reproduce a CodeBuild build, on your machine and really see what’s going on when you have failures.
Inside VPC
Also CodeBuild can be launched within the VPC.
So by default, you could build containers, instances are launched outside your VPC. That means that it’s going to run fine, but it cannot access some resources that are within your VPC. So you can specify a VPC configuration for CodeBuild with a VPC ID, subnet IDs, Security Group IDs, and so on.
And then thanks to this then CodeBuild containers will be able to access resources in your VPC, such as RDS, ElastiCache, EC2 instances, ALBs and so on. So in this example, I’ll have RDS database in a private subnet in my VPC, and then I can directly launch my CodeBuild container in here.
And then my CodeBuild container could access my RDS database instance if we needed to. So to use cases to have CodeBuild inside your VPC, is to do integration testing, data query, to talk to internal load balancers and so on.
Environment Variables
Okay, so a few more things you need to know about CodeBuild.
The first one is around Environment Variables. So we know that there are some variables in our CodeBuild project at runtime, but what are they?
The first category is Default Environment Variables. As you can see, they’re defined and provided by AWS for every of your build. And they include, for example, the AWS default region, the build ARN, the build ID, the build image, and a lot of metadata around your build.
The second is Custom Environmental Variables. So they have two kinds. The first one are static and static are defined at build time. So you can say, for example, that this build is for the environment production or whatever and you can override them when you use the start-build API call if you need it to.
The last kind is Dynamic Environment Variables. Here, we are pulling the value of the environment variable through the SSM Parameter Store or Secrets Manager. So this is an integration, for example, we have a secret value, we cannot pass it statically, but we can, for example, have CodeBuild retrieve the value of the secret directly from Secrets Manager at runtime.
Security
So what about security of CodeBuild? So CodeBuild has a service role, and this service role is used to access AWS resources on your behalf.
So we assign a role and, for example, we can download code from CodeCommit. We can fetch parameters from the SSM Parameter Store or we can, for example, upload artifacts to S3 Buckets, all the things this service role can help us do, for example, is fetching secrets from Secrets Manager, storing logs and CloudWatch logs and much more.
So on top of it, we have in-transit and at-rest data encryption for your cache, for your logs, et cetera, et cetera. And any build output artifact is going to be encrypted and that requires access to KMS.
Build Badges
Okay, so what about build badges?
So for example, we are in CodeCommit and we want to have a look at whether or not our pull request is passing or not.
So here is an example where there’s a comment on the CodeCommit pull request where the build is passing.
And here is an example where the build is failing.
So this passing and failing little image is called a Build Badge.
So these badges are dynamically generated and they display the status of the latest build in CodeBuild. And so that can be accessed through a public URL for this build badge.
And it’s supported for CodeCommit, GitHub and BitBuckets.
Now these badges are available at the branch level and so that means that if you are doing a new pull request on the specific branch, then you can have a CodeBuild related to that branch and then the build will display some information about the status of the branch itself.
Triggers
Now what about CodeBuild triggers? So how do we trigger a CodeBuild?
So multiple ways, we can have CodeCommit push an event into EventBridge and that event may trigger directly CodeBuild. So there is an integration between EventBridge and CodeBuild.
We can have CodeCommit as well push an event to EventBridge. We can then trigger Lambda function, do something a little bit more complicated and then from Lambda, use the API call to invoke CodeBuild.
We can also, for example, use GitHub because CodeBuild can work at GitHub. In that case, GitHub will push an event into a Web Hook for CodeBuild and this web hook will trigger a CodeBuild build.
Validate Pull Requests
So all these things are possible and the reason I’m telling you this is that because we have a architecture we can use to validate pull request in CodeCommits.
So how do we do this?
Well, for example, we say we have pull request in CodeCommit, so there is new proposed code changes, but we want to validate them. We want to test them before they get merged to make sure that the build is working. This allows us to ensure a high-level of code quality and avoid code conflicts and then as a result, we could also display the badge on your PR.
So how does that work?
Well, we have a CodeCommit repository with several branches and we want to merge development branch into production branch. So for this, we’re going to create a pull request and then whenever that pull request is created or updated, then an event is going to be published into EventBridge. From there we can trigger a first Lambda function whose role is going to update the pull request with a comment saying that the test build has begun.
And EventBridge as a second destination, which is going to be CodeBuild. So CodeBuild is going to be invoked on that specific PR and so it’s going to check out the code, do a test, and then either the test are, and the build is successful or not. So that is going to be published to EventBridge which can then invoke Lambda function which knows how to parse these events.
And then is going to update the pull request with a comment with the build outcome. And then we’re good to go. We can actually merge that pull request if we’re happy with it.
So visually, here’s an example of this architecture.
We have a first comment in the bottom that says that the build has been started and then we have a second comment on the top saying that the build has been failing. And on top of it, we add the build batch to say it was failed.
In case the pull request is passing, we have a comment saying that the CodeBuild is also passing, and so we’re good to go.
We know we can safely merge that pull request into our master branch.
Test Reports
So CodeBuild is running a lot of tests and there’s a way for you to get a visual test report directly from the CodeBuild UI. And so as such, this is called a report groups and test reports.
And so as you can see in this UI, we have a 75% pass rate for the test rerun within CodeBuild. And so that gives us the idea of what is failing or what is not failing. We don’t have to go through logs to determine this.
We don’t have to go through logs to determine this. So how do we go to create these test reports?
Well, they contain details about the test that are run during builds. And so this could be any kind of test such as unit test, configuration test, and functional test. And you can create your test cases with any test framework that will create a report file, and then there is some format that CodeBuild can understand.
So we have the JUnit XML, the NUnit XML, the NUnit3 XML. We have the Cucumber JSON, the TestNG XML, and the Visual Studio TRX.
And so the idea is that whatever framework you use as long as the report is one of these formats, for example, the Cucumber JSON, then automatically, you can create a test reports directly in CodeBuild.
So the way to do it is that in your buildspec.yml, which indicates how to function to your CodeBuild, then there’s just a report group and you define where the files that represent your reports are located on your build and automatically they will surface back into CodeBuild, which is quite nice.
