If you are just getting started with JavaScript, the number of tools and technologies you'll hear about may be overwhelming. And you might have a hard time deciding which tools you actually need.

Or maybe you're familiar with the tools, but you haven't given much thought to what problems they solve and how miserable your life would be without their help.

I believe it is important for Software Engineers and Developers to understand the purpose of the tools we use every day.

That's why, in this article, I look at NPM, Babel, Webpack, ESLint, and CircleCI and I try to clarify the problems they solve and how they solve them.

NPM

NPM is the default package manager for JavaScript development. It helps you find and install packages (programs) that you can use in your programs.

You can add npm to a project simply by using the "npm init" command. When you run this command it creates a "package.json" file in the current directory. This is the file where your dependencies are listed, and npm views it as the ID card of the project.

You can add a dependency with the "npm install (package_name)" command.

When you run this command, npm goes to the remote registry and checks if there is a package identified by this package name. If it finds it, a new dependency entry is added to your package.json and the package, with it's internal dependencies, is downloaded from the registry.

You can find downloaded packages or dependencies under the "node_modules" folder. Just keep in mind that it usually gets pretty big – so make sure to add it to .gitignore.

NPM does not only ease the process of finding and downloading packages but also makes it easier to work collaboratively on a project.

Without NPM, it would be hard to manage external dependencies. You would need to download the correct versions of every dependency by hand when you join an existing project. And that would be a real hassle.

With the help of npm, you can just run "npm install" and it will install all external dependencies for you. Then you can just run it again anytime someone on your team adds a new one.

Babel

Babel is a JavaScript compiler or transpiler which translates the ECMAScript 2015+ code into code that can be understood by older JavaScript engines.

Babel is the most popular Javascript compiler, and frameworks like Vue and React use it by default. That said, concepts we will talk about here are not only related to Babel and will apply to any JavaScript compiler.

Why do you need a compiler?

"Why do we need a compiler, isn't JavaScript an interpreted language?" you may ask if you are familiar with the concepts of compiled and interpreted languages.

It's true that we usually call something a "compiler" if it translates our human-readable code to an executable binary that can be understood by the CPU. But that is not the case here.

The term transpiler may be more appropriate since it is a subset of a compiler: Transpilers are compilers that translate the code from a programming language to another language (in this example, from modern JS to an older version).

JavaScript is the language of browsers. But there is a problem with browsers: Cross compatibility. JavaScript tools and the language itself are evolving rapidly and many browsers fail to match that pace. This results in compatibility issues.

You probably want to write code in the most recent versions of JavaScript so you can use its new features. But if the browser that your code is running has not implemented some of the new features in its JavaScript engine, the code will not execute properly on that browser.

This is a complex problem because every browser implements the features at a different speed. And even if they do implement those new features, there will always be people who use an older version of their browser.

So what if you want to be able to use the recent features but also want your users to view those pages without any problems?

Before Babel, we used polyfills to run older versions of certain code if the browser did not support the modern features. And when you use Babel, it uses polyfills behind the scenes and does not require you to do anything.

How do transpilers/compilers work?

Babel works similar to other compilers. It has parsing, transformation, and code generation stages.

We won't go in-depth here into how it works, since compilers are complicated things. But to understand the basics of how compilers work, you can check out the the-super-tiny-compiler project. It is also mentioned in Babel's official documentation as being helpful in understanding how Babel works.

We can usually get away with knowing about Babel plugins and presets. Plugins are the snippets that Babel uses behind the scenes to compile your code to older versions of JavaScript. You can think of each modern feature as a plugin. You can go to this link to check out the full list of plugins.

List of plugins for ES5

Presets are collections of plugins. If you want to use Babel for a React project you can use the pre-made @babel/preset-react which contains the necessary plugins.

React Preset Plugins

You can add plugins by editing the Babel config file.

Do you need Babel for your React App?

For React, you need a compiler because React code generally uses JSX and JSX needs to be compiled. Also the library is built on the concept of using ES6 syntax.

Luckily, when you create a project with create-react-app, it comes with Babel already configured and you usually do not need to modify the config.

Examples of a compiler in action

Babel's website has an online compiler and it is really helpful to understand how it works. Just plug in some code and analyze the output.

Webpack

Webpack is a static module bundler. When you create a new project, most JavaScript frameworks/libraries use it out of the box nowadays.

If the phrase "static module bundler" sounds confusing, keep reading because I have some great examples to help you understand.

Why do you need a bundler?

In web apps you're going to have a lot of files. This is especially the case for Single Page Applications (React, Vue, Angular), with each having their own dependencies.

What I mean by a dependency is an import statement – if file A needs to import file B to run properly, then we say A depends on B.

In small projects, you can handle the module dependencies with <script> tags. But when the project gets larger, the dependencies rapidly become hard to manage.

Maybe, more importantly, dividing the code into multiple files makes your website load more slowly. This is because the browser needs to send more requests compared to one large file, and your website starts to consume a ton of bandwidth, because of HTTP headers.

We, as developers want our code to be modular. We divide it into multiple files because we do not want to work with one file with thousands of lines. Still, we also want our websites to be performant, to use less bandwidth, and to load fast.

So now, we'll see how Webpack solves this issue.

How Webpack works

When we were talking about Babel, we mentioned that JavaScript code needs to be transpiled before the deployment.

But compiling with Babel is not the only operation you need before deploying your project.

You usually need to uglify it, transpile it, compile the SASS or SCSS to CSS if you are using any preprocessors, compile the TypeScript if you are using it...and as you can see, this list can get long easily.

You do not want to deal with all those commands and operations before every deployment. It would be great if there was a tool that did all that for you in the correct order and correct way.

The good news – there is: Webpack.

Webpack also provides features like a local server with hot reload (they call it hot module replacement) to make your development experience better.

So what's hot reloading? It means that whenever you save your code, it gets compiled and deployed to the local HTTP server running on your machine. And whenever a file changes, it sends a message to your browser so you do not even need to refresh the page.

If you have ever used "npm run serve", "npm start" or "npm run dev", those commands also start Webpack's dev server behind the scenes.

Webpack starts from the entry point of your project (index) and generates the Abstract Syntax Tree of the file. You can think of it as parsing the code. This operation is also done in compilers, which then look for import statements recursively to generate a graph of dependencies.

It then converts the files into [IIFEs](https://developer.mozilla.org/en-US/docs/Glossary/IIFE#:~:text=An%20IIFE%20(Immediately%20Invoked%20Function,soon%20as%20it%20is%20defined.) to modularize them (remember, putting code inside a function restricts its scope). By doing this, they modularize the files and make sure the variables and functions are not accessible to other files.

Without this operation, it would be like copying and pasting the code of the imported file and that file would have the same scope.

Webpack does many other advanced things behind the scenes, but this is enough to understand the basics.

Bonus – ESLint

Code quality is important and helps keep your projects maintainable and easily extendable. While most of us developers recognize the significance of clean coding, we sometimes tend to ignore the long term consequences under the pressure of deadlines.

Many companies decide on coding standards and encourage developers to obey those standards. But how can you make sure that your code meets the standards?

Well, you can use a tool like ESLint to enforce rules in the code. For example, you can create a rule to enforce or disallow the usage of semicolons in your JavaScript code. If you break a rule, ESLint shows an error and the code does not even get compiled – so it is not possible to ignore that unless you disable the rule.

Linters can be used to enforce standards by writing custom rules. But you can also use the pre-made ESLint configs established by big tech companies to help devs get into the habit of writing clean code.

You can take a look at Google's ESLint config here – it is the one I prefer.

ESLint helps you get used to best practices, but that's not its only benefit. ESLint also warns you about possible bugs/errors in your code so you can avoid common mistakes.

Bonus – CI/CD (CircleCI)

Continuous Integration/Development has gained a lot of popularity in recent years as many companies have adopted Agile principles.

Tools like Jenkins and CircleCI allow you to automate the deployment and testing of your software so you can deploy more often and reliably without going through difficult and error-prone build processes by yourselves.

I mention CircleCI as the product here because it is free and used frequently in JavaScript projects. It's also quite easy to use.

Let's go over an example: Say you have a deployment/QA server and your Git repository. You want to deploy your changes to your deployment/QA server, so here is an example process:

1. Push the changes to Git
2. Connect to the server
3. Create a Docker container and run it
4. Pull the changes to the server, download all the dependencies (npm install)
5. Run the tests to make sure nothing is broken
6. Use a tool like ESLint/Sonar to ensure code quality
7. Merge the code if everything is fine

With the help of CircleCI, you can automatically do all these operations. You can set it up and configure to do all of the above operations whenever you push a change to Git. It will reject the push if anything goes wrong, for example a failing test.

I will not get into the details of how to configure CircleCI because this article is more about the "Why?" of each tool. But if you are interested in learning more and seeing it in action, you can check out this tutorial series.

Conclusion

The world of JavaScript is evolving rapidly and new tools are gaining popularity every year.

It's easy to react to this change by just learning how to use the tool – we are often too busy to take our time and think about the reason why that tool became popular or what problem it solves.

In this article, I picked the tools I think are most popular and shared my thoughts on their significance. I also wanted to make you think about the problems they solve rather than just the details of how to use them.

If you liked the article you can check out and subscribe to my blog where I try to write frequently. Also, let me know what you think by commenting so we can brainstorm or you can tell me what other tools you love to use :)

CircleCI is a popular tool for orchestrating CI/CD pipelines. Lighthouse is an open-source project from Google for improving the quality of web pages. It provides user-centric metrics to audit SEO, performance, accessibility, best practices, and progressive web apps.

For a deeper dive about Lighthouse you can read “How to analyze website performance with Lighthouse”.

By combining these forces, we can establish website quality automation. This post will demonstrate the following:

• Basic implementation of Lighthouse in a CircleCI workflow.
• Advanced setup to display Lighthouse results in pull request comments.
• S3 Lighthouse report uploads.
• Slack notifications.

Lighthouse Check CircleCI Orb

CircleCI Orbs are shareable packages of configuration elements, including jobs, commands, and executors you use in your workflows. This post will provide a guide for using the Lighthouse Check CircleCI Orb for automating Lighthouse audits.

Basic Example

Below is a minimal example and all we need to run Lighthouse automatically on any code change ? In this example both https://www.foo.software and https://www.foo.software/contact will be audited.

version: 2.1

orbs:
  lighthouse-check: foo-software/lighthouse-check@0.0.8

jobs:
  test: 
    executor: lighthouse-check/default
    steps:
      - lighthouse-check/audit:
          urls: https://www.foo.software,https://www.foo.software/contact

workflows:
  test:
    jobs:
      - test

With this minimal setup we have a summary provided in the output of our job. We also have full HTML reports saved as CircleCI "artifacts".

Lighthouse Check Orb output

Lighthouse Check Orb save artifacts

Advanced Example

Lighthouse Check CircleCI Orb offers a complete set of bells and whistles by utilizing [lighthouse-check](https://github.com/foo-software/lighthouse-check) NPM module under the hood. There’s so much more we can do with this. Let’s proceed!

Pull Request Comments

By utilizing this feature, comments are posted with Lighthouse results on every commit. We can do so by following the steps below.

1. Create a new user or find an existing one to act as a “bot”.
2. Create a personal access token from this user account.
3. Create a CircleCI environment variable in your project to hold the encrypted value from above. In our example we name it LIGHTHOUSE_CHECK_GITHUB_ACCESS_TOKEN.
4. Update our config file with the diff shown below.

+ prCommentAccessToken: $LIGHTHOUSE_CHECK_GITHUB_ACCESS_TOKEN
+ prCommentUrl: https://api.github.com/repos/foo-software/lighthouse-check-orb/pulls/${CIRCLE_PULL_REQUEST##*/}/reviews
urls: https://www.foo.software,https://www.foo.software/contact

The updates above provides a token to authorize comments on a corresponding pull request. prCommentUrl should be an endpoint in the format specified by the GitHub API. Your endpoint will be similar but with owner and repo params replaced ( foo-software/lighthouse-check-orb ). With this, we’ve created a bot to post Lighthouse results on pull requests ?

Lighthouse Check PR comments

S3 Report Uploads

In our example we persist results by uploading reports as artifacts in our job. This solution could be sufficient in some cases, but artifacts aren’t stored permanently. In order to view reports, we need to navigate into the workflow and manually download reports from the artifact view.

But what if we want a more dependable way of storing and referencing reports? This is where the S3 feature comes into play. We can configure AWS S3 storage by following the below steps.

1. Create an AWS account if you don’t already have one.
2. Create an S3 bucket if you don’t already have one.
3. Acquire an AWS access key id and secret access key.
4. Create a CircleCI environment variables for these two values. In our example we’ll use LIGHTHOUSE_CHECK_AWS_ACCESS_KEY_ID and LIGHTHOUSE_CHECK_AWS_SECRET_ACCESS_KEY, respectively.
5. Add the bucket name and region (example: us-east-1) as CircleCI environment variables: LIGHTHOUSE_CHECK_AWS_BUCKET and LIGHTHOUSE_CHECK_AWS_REGION.

Next, we’ll update our configuration with the following diff.

+ awsAccessKeyId: $LIGHTHOUSE_CHECK_AWS_ACCESS_KEY_ID
+ awsBucket: $LIGHTHOUSE_CHECK_AWS_BUCKET
+ awsRegion: $LIGHTHOUSE_CHECK_AWS_REGION
+ awsSecretAccessKey: $LIGHTHOUSE_CHECK_AWS_SECRET_ACCESS_KEY
prCommentUrl: https://api.github.com/repos/foo-software/lighthouse-check-orb/pulls/${CIRCLE_PULL_REQUEST##*/}/reviews

In our next commit and push, reports are automatically uploaded to S3 ✅! We also have a them linked in our PR comments.

Lighthouse result PR comment with S3 report linked

Slack Notifications

What’s a new feature in a DevOps workflow without Slack notifications? A sad one indeed. Let’s ramp things up by adding notifications to a Slack channel for our whole team to see. We can accomplish this in the below steps.

1. Create an “Incoming Webhook” in your Slack workspace and authorize a channel.
2. Add the Webhook URL as a CircleCI environment variable — LIGHTHOUSE_CHECK_SLACK_WEBHOOK_URL.

+ slackWebhookUrl: $LIGHTHOUSE_CHECK_SLACK_WEBHOOK_URL
urls: https://www.foo.software,https://www.foo.software/contact

Our next commit and push introduces Slack notifications! The “Lighthouse audit” link in the below screenshot navigates to the S3 report as configured ✨

Lighthouse Slack notification

At this point our complete "advanced example" configuration looks like the below.

usage:
  version: 2.1

  orbs:
    lighthouse-check: foo-software/lighthouse-check@0.0.8

  jobs:
    test: 
      executor: lighthouse-check/default
      steps:
        - lighthouse-check/audit:
            awsAccessKeyId: $LIGHTHOUSE_CHECK_AWS_ACCESS_KEY_ID
            awsBucket: $LIGHTHOUSE_CHECK_AWS_BUCKET
            awsRegion: $LIGHTHOUSE_CHECK_AWS_REGION
            awsSecretAccessKey: $LIGHTHOUSE_CHECK_AWS_SECRET_ACCESS_KEY
            prCommentAccessToken: $LIGHTHOUSE_CHECK_GITHUB_ACCESS_TOKEN
            prCommentUrl: https://api.github.com/repos/foo-software/lighthouse-check-orb/pulls/${CIRCLE_PULL_REQUEST##*/}/reviews
            slackWebhookUrl: $LIGHTHOUSE_CHECK_SLACK_WEBHOOK_URL
            urls: https://www.foo.software,https://www.foo.software/contact

  workflows:
    test:
      jobs:
        - test

Maintaining a Historical Record

Foo’s Automated Lighthouse Check is tool we can use to manage a historical record of Lighthouse audits. We can connect to it with the Lighthouse Check Orb! By doing so you can run Lighthouse remotely versus in a local, dockerized CircleCI environment. With this we can be assured our Lighthouse results aren't flaky from CircleCI infrastructure change. Follow the documented steps to connect with Automated Lighthouse Check.

A historical record of Lighthouse audits with "Automated Lighthouse Check"

What Now?

You can find other of examples of Lighthouse Check Orb usage on GitHub. I hope this article has provided a helpful addition to your DevOps workflow! With Lighthouse integrated in a CI/CD pipeline, we can stay fully equipped to ensure high quality in website SEO, performance, accessibility, best practice, and progressive web apps.

Consumer driven contract testing is a great way to improve the reliability of interconnected systems. Integration testing becomes way easier and more self contained. It opens the door for independent deployments, and leads to faster iterations and more granular feedback. Unlike your insurance, it doesn't have any fine print. This article is about setting it up in a delivery pipeline, in the context of doing continuous delivery.

I want to show how Contract Tests help split the deployment of the front end and the back end of a small application. I have a React client and a Spring Boot backend written in Kotlin.

What is a Contract Test?

I am not talking about smart contracts. There is no blockchain whatsoever in this article. Sorry for that (Contract Tests for Smart Contracts sounds like a conference talk that the world badly needs, though!).

In a nutshell, a Contract Test is a specification of the interactions between a consumer and a provider. In our case, the communication happens using REST. The consumer defines the actions sent to the provider and the responses that will be returned. In our case, the frontend is the consumer and the backend is the provider. A contract is generated. Both sides test against this contract.

It is not really about any particular technology. There are a bunch of different frameworks, but some simple scripts could do the trick.

Why have it as part of the delivery pipeline?

First of all, running these tests continuously ensures that they keep working at all times. The big benefit, however, is that we can separate the deployment of the front end and back end. If both sides are fulfilling the contract, it is likely that they work together correctly. Thus, we can consider avoiding expensive integrated tests. They tend to work pretty badly anyways.

Setting up some contracts

There are two sides to set up, consumer and provider. The tests will run in the pipelines that build the front end and the back end, respectively. We are going to use the Pact framework for our examples, which is the tool that I am most familiar with. Because of that, I tend to use pact and contract interchangeably. Our pipelines are written for CircleCI, but they should be fairly easy to port to other CI Tools.

The consumer side

As mentioned, the consumer leads the creation of the contract. Having the client driving this might sound counterintuitive. Often, APIs are created before the clients that will use them. Flipping it around is a nice habit to get into. It forces you to really think in terms of what the client will actually do, instead of bikeshedding a super generic API that will never need most of its features. You should give it a try!

The pact is defined through interactions specified in unit tests. We specify what we expect to be sent to the back end, and then use the client code to trigger requests. Why? We can compare expectations against actual requests, and fail the tests if they don't match.

Let's have a look at an example. We are using Jest to run the tests. We'll start with some initialization code:

import path from 'path'
import Pact from 'pact'

const provider = () =>
  Pact({
    port: 8990,
    log: path.resolve(process.cwd(), 'logs', 'pact.log'),
    dir: path.resolve(process.cwd(), 'pacts'),
    spec: 2,
    consumer: 'frontend',
    provider: 'backend'
  })

export default provider

Then we have the code for an actual test. The test consists of two parts. First we define the expected interaction. This is not very different from mocking an http library, with something like axios. It specifies the request that we will send (URL, headers, body and so forth), and the response that we will get.

const interaction: InteractionObject = {
  state: 'i have a list of recipes',
  uponReceiving: 'a request to get recipes',
  withRequest: {
    method: 'GET',
    path: '/rest/recipes',
    headers: {
      Accept: 'application/json',
      'X-Requested-With': 'XMLHttpRequest'
    }
  },
  willRespondWith: {
    status: 200,
    headers: { 'Content-Type': 'application/json; charset=utf-8' },
    body: [
      {
        id: 1,
        name: 'pasta carbonara',
        servings: 4,
        duration: 35
      }
    ]
  }
}

Then we have the test itself, where we call the actual client code that will trigger the request. I like to encapsulate these requests in services that convert the raw response into the domain model that will be used by the rest of the app. Through some assertions, we make sure that the data that we are delivering from the service is exactly what we expect.

it('works', async () => {
  const response = await recipeList()

  expect(response.data.length).toBeGreaterThan(0)
  expect(response.data[0]).toEqual({
    id: 1,
    name: 'pasta carbonara',
    servings: 4,
    duration: 35
  })
})

Note that even if recipeList is properly typed with TypeScript, that won't help us here. Types disappear at runtime, so if the method is returning an invalid Recipe we won't realize it, unless we explicitly test for it.

Finally we need to define some extra methods that will ensure that the interactions are verified. If there are interactions missing, or they don't look like they should, the test will fail here. After that, all that remains is writing the pact to disk.

beforeAll(() => provider.setup())
afterEach(() => provider.verify())
afterAll(() => provider.finalize())

In the end, the pact gets generated as a JSON file, reflecting all the interactions that we have defined throughout all our tests.

Flexible matching

Our pact thus far is specifying the exact values that it will get from the backend. That won't be maintainable in the long run. Certain things are inherently harder to pin down to exact values (for example, dates).

A pact that breaks constantly will lead to frustration. We are going through this whole process to make our life easier, not harder. We'll avoid that by using matchers. We can be more flexible and define how things will look like, without having to provide exact values. Let's rewrite our previous body:

willRespondWith: {
  status: 200,
  headers: { 'Content-Type': 'application/json; charset=utf-8' },
  body: Matchers.eachLike({
    id: Matchers.somethingLike(1),
    name: Matchers.somethingLike('pasta carbonara'),
    servings: Matchers.somethingLike(4),
    duration: Matchers.somethingLike(35)
  })
}

You can be more specific. You can set the expected length of a list, use regexes and a bunch of other things.

Integrating it in the pipeline

The pact tests rely on an external process, and having multiple tests hitting it can lead to non deterministic behavior. One solution is to run all the tests sequentially:

npm test --coverage --runInBand

If you want to run the pact tests independently, we can build our own task to run them separately:

"scripts": {
  "pact": "jest --transform '{\"^.+\\\\.ts$\": \"ts-jest\"}' --testRegex '.test.pact.ts$' --runInBand"
}

Which will become an extra step in our pipeline:

jobs:
  check:
    working_directory: ~/app

    docker:
      - image: circleci/node:12.4

    steps:
      - checkout
      - run: npm
      - run: npm run linter:js
      - run: npm test --coverage --runInBand
      - run: npm pact

Storing the pact

Our pact is a json file that we are going to commit directly in the frontend repository, after running the tests locally. I've found that this tends to work well enough. Making the pipeline itself commit the pact to git does not seem to be necessary.
We'll get to extending the pact and in a second.

The provider side

At this point we have a working pact, that is being verified by the consumer. But that is only half of the equation. Without a verification from the provider side, we haven't accomplished anything. Maybe even less than that, because we might get a false sense of security!

To do this, we are going to start the back end as a development server and run the pact against it. There is a gradle provider that takes care of this. We need to configure it and provide a way of finding the pact (which is stored in the front end repository). You can fetch the pact from the internet, or from a local file, whichever is more convenient.

buildscript {
    dependencies {
        classpath 'au.com.dius:pact-jvm-provider-gradle_2.12:3.6.14'
    }
}

apply plugin: 'au.com.dius.pact'

pact {
    serviceProviders {
        api {
            port = 4003

            hasPactWith('frontend') {
                pactSource = url('https://path-to-the-pact/frontend-backend.json')
                stateChangeUrl = url("http://localhost:$port/pact")
            }
        }
    }
}

What remains is starting the server and running the pact against it, which we do with a small script:

goal_test-pact() {
  trap "stop_server" EXIT

  goal_build
  start_server

  ./gradlew pactVerify
}

start_server() {
  artifact=app.jar
  port=4003

  if lsof -i -P -n | grep LISTEN | grep :$port > /dev/null ; then
    echo "Port[${port}] is busy. Server won't be able to start"
    exit 1
  fi

  nohup java -Dspring.profiles.active=pact -jar ./build/libs/${artifact} >/dev/null 2>&1 &

  # Wait for server to answer requests
  until curl --output /dev/null --silent --fail http://localhost:$port/actuator/health; do
    printf '.'
    sleep 3
  done
}

stop_server() {
  pkill -f 'java -Dspring.profiles.active=pact -jar'
}

Fixtures

If you are running your back end in development mode, it will have to deliver some data, so that the contract is fulfilled. Even if we are not using exact matching, we have to return something, otherwise it won't be possible to verify it.

You can use mocks, but I've found that avoiding them as much as possible leads to more trustworthy results. Your app is closer to what will happen in production. So what other options are there? Remember that when we were defining interactions, we had a state. That's the cue for the provider. One way of using it is the stateChangeUrl. We can provide a special controller to initialize our back end based on the state:

private const val PATH = "/pact"

data class Pact(val state: String)

@RestController
@RequestMapping(PATH, consumes = [MediaType.APPLICATION_JSON_VALUE])
@ConditionalOnExpression("\${pact.enabled:true}")
class PactController(val repository: RecipeRepository) {
    @PostMapping
    fun setup(@RequestBody body: Pact): ResponseEntity<Map<String,String>> {
        when(body.state) {
            "i have a list of recipes" -> initialRecipes()
            else -> doNothing()
        }

        return ResponseEntity.ok(mapOf())
    }
}

Note that this controller is only active for a specific profile, and won't exist outside of it.

Integrating it in the pipeline

As with the provider, we will run the check as part of our pipeline

version: 2
jobs:
  build:

    working_directory: ~/app

    docker:
      - image: circleci/openjdk:8-jdk

    steps:

      - checkout
      - run: ./go linter-kt
      - run: ./go test-unit
      - run: ./go test-pact

There is a slight difference, though. Our contract gets generated by the consumer. That means that a change in the front end could lead to a pact that does not verify properly anymore, even though no code was changed in the back end. So ideally, a change in the pact should trigger the back end pipeline as well. I haven't found a way to represent this elegantly in CircleCI, unlike say ConcourseCI.

How the contract influences the relationship between front end and back end

It's nice that we got this set up. Never touch a running system, right? Well, we might! After all, quick change is why we invest in all this tooling. How would you introduce a change that requires extending the API?

1. We start with the client. We want to define what the client will get that is not there yet. As we learned, we do that through a test in the front end that defines the expectation for the new route, or the new fields. That will create a new version of the pact.
2. Note that at this point the back end does not fulfill the pact. A new deployment of the back end will fail. But also, the existing backend does not fulfill the pact either right now. The change you introduced has to be backwards compatible. The front end should not be relying on the changes, either.
3. Now it's time to fulfill the new pact from the back end side. If this takes a long time, you will block your deployment process, which is not good. Consider doing smaller increments in that case. Anyways, you've got to implement the new functionality. The pact test will verify that your change is what's actually expected.
4. Now that the back end is providing the new functionality, you can freely integrate it in your front end.

This flow can get a bit awkward in the beginning. It is really important to work with the smallest quantum of functionality. You don't want to block your deployment process.

Next steps

For the integration between your own front end and back end I have found this setup to be sufficient in practice. However, as complexity grows, versioning will become important. You'll want to help multiple teams collaborate more easily. For that, we can use a broker. This is a lot harder to implement, so you should ask yourself if you really need it. Don't fix problems that you don't have yet.

Conclusion

To summarize, this is the setup we arrived at:

Think about all the time you have spent writing tests to check that your back end is sending the right data. That is a lot more convenient to do with a contract. Moreover, releasing front end and back end independently means being faster, releasing smaller pieces of functionality. It might feel scary at first, but you will realize that you actually are much more aware of what's going out that way.

Once you have adopted this for one service, there is no reason not to do it for all of them. I really don't miss running costly end to end test suites just to verify that my back end is working. Here is the code that I used in the examples for the front end and the back end. It is a full running (albeit small) application. Good luck with your contracts!

There are many deployment solutions out there with deploying Laravel, ranging from SSH’ing into your machine and git pulling the files (?) to time-savers like Envoyer or rolling your own with Deployer. I personally love Deployer as it is free, very flexible and can support your projects from infancy to when you scale to multiple machines.

Deployer works by running your deployment scripts locally and we can make this even more convenient by integrating Deployer into your CI pipeline. For a recent project, I used CircleCI to run all my tests and automatically deploy.

Setup Deployer Locally

To save time, I will highly recommend that you have Deployer installed locally and configured and working for your app. This saves a lot of time in having to do debugging on your Deployer config when your CircleCI build fails.

Laravel and CircleCI

If you already have CircleCI setup for your Laravel project tests, skip to the next section. Otherwise, I assume that you at least already have a CircleCI account and a basic project created inside.

Create a .circleci/config.yml and with the below

version: 2
jobs:
  build:
    docker:
      # Specify the version you desire here
      - image: circleci/php:7.1-browsers
      - image: circleci/mysql:5.7
        environment:
          MYSQL_ALLOW_EMPTY_PASSWORD: yes
          MYSQL_USER: root
          MYSQL_ROOT_PASSWORD: ''
          MYSQL_DATABASE: laravel

    working_directory: ~/laravel

    steps:
      - checkout
      - run:
          name: Install PHP exts
          command: |
            sudo docker-php-ext-install zip
            sudo docker-php-ext-install pdo_mysql
            sudo apt install -y mysql-client
      - run: sudo composer self-update

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "composer.json" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run: composer install -n --prefer-dist

      - save_cache:
          paths:
            - ./vendor
          key: v1-dependencies-{{ checksum "composer.json" }}

      - run:
          name: Setup Laravel stuffs
          command: |
            php artisan migrate --force
      - run: ./vendor/bin/phpunit

workflows:
  version: 2
  notify_deploy:
    jobs:
      - build

This is a very basic config.yml for Laravel. It will install PHP, MySQL and run PHPunit tests. Feel free to modify this to suit your needs.

CircleCI Deployment

After our tests are run, we want CircleCI to automatically start deploying to our server. Remember that Deployer uses SSH, so in our remote server, we want to create a SSH keypair ssh-keygen -t rsa -b 4096 -C “your@email.com". We want to add this key we created into CircleCI as a deployment key. Take note of the fingerprint, as we will need to add this into our config file later.

Let’s create a new job in our config file:

jobs:
  build: ... # from above

  deploy:
    docker:
      - image: circleci/php:7.2-browsers
    working_directory: ~/laravel
    steps:
      - checkout

This tells CircleCI that we have a new job called deploy and to build it based on the php-7.2-browsers docker image.

Remember the fingerprint that we took note of when we added it into CircleCI? We need to reference this in in our config so that the SSH key will be present in our container.

jobs:
  build: ... # from above

  deploy:
    docker:
      - image: circleci/php:7.2-browsers
    working_directory: ~/laravel
    steps:
      - checkout
      - add_ssh_keys:
          fingerprints:
            - "YOUR_FINGERPRINT"

So now, CircleCI knows to add this key via the fingerprint into the docker container and this will allow us to run Deployer and SSH into our remote server to deploy.

jobs:
  build: ... # from above

  deploy:
    docker:
      - image: circleci/php:7.2-browsers
    working_directory: ~/laravel
    steps:
      - checkout
      - add_ssh_keys:
          fingerprints:
            - "YOUR_FINGERPRINT"
      - run:
          name: Install Deployer
          command: |
            curl -LO https://deployer.org/deployer.phar
            sudo mv deployer.phar /usr/local/bin/dep
            sudo chmod +x /usr/local/bin/dep
      - run:
          name: Deploy
          command: |
            dep deploy www.your_server.com

With the deploy job defined, the last step would be to tell CircleCI to run it after our tests pass. That’s simple, we already have a basic workflow named notify_deploy that we can build upon.

workflows:
  version: 2
  notify_deploy:
    jobs:
      - build
      - deploy:
          requires:
            - build
          filters:
            branches:
              only: master

This tells CircleCI that in our notify_deploy workflow, aside from running build, we want to run deploy after it is finished, and to only do it on the master branch.

With everything done, we can now push the config file to our git repository and watch it test and deploy on its own. Do you use a different way of deploying Laravel? Let me know, I’d love to hear about it!

Continuous Deployment might seem complicated at first, but don’t be intimidated. In this tutorial, I’ll show you how to implement Continuous Deployment to AWS S3 for a static website using CircleCI in less than 30 minutes.

You’ll need both an AWS account and a CircleCI account. If you don’t have these yet, start by opening a free account for AWS here and a free CircleCI account here . Both AWS and CircleCI have a free tier that’s more than enough for what you will need for this tutorial.

Getting the code

First you will start by forking and cloning the following project repo on Github: S3ContinuousDeploy or if you prefer you can try this tutorial with one of your own repos as long as it’s a static site.

Next you will add the project to your CircleCI account.

Next select the S3ContinuousDeploy repo you just cloned and click build project.

Choose the S3ContinuousDeploy repo and click build project

At this point the build will run, but you will be getting an error message warning you that the settings for your project couldn’t be detected. Which is normal since we don’t have a circle.yml configuration file in place, which is what you will be doing next.

Looking at the docs at CircleCI you can get an idea of what the circle.yml should look like. Unfortunately the circle.yml file example provided will not work as is and will need some tweaking, so let’s do that.

Below is the modified circle.yml file you will be using:

Basically CircleCI creates the build within a Docker container, and the override under dependencies property (line 3) that I added instructs CircleCI to install the AWS command line interface (awscli) that will be used in this case to help manage and facilitate deployment to AWS S3.

So make sure you add the file and commit it to your repo. Finally make sure you push this and other commits you might have made before you proceed to the next step.

As per CircleCI documentation the command for deploying is:

The path-to-file was a bit tricky to figure out but by looking at the error logs I was able to finally get it right: home\ubuntu\projectName. So just replace projectName with the name of your project, in my case that will be S3ContinuousDeploy.

The S3://bucket-URL on the other hand is not correct and should be S3://bucket-Name. Right now we don’t have a bucket name, so let’s get ourselves a bucket.

Creating the S3 bucket

In this step we will be heading to the AWS Console to create the S3 bucket for this project:

In Your Console go to Storage and then to S3

Press Create bucket

Enter the bucket name you would like to use for this project as well as the region. (The best practice is to use the region nearest to your site’s audience.)

You will skip the other steps for now so press “Next” and then press “Create bucket” on the review screen.

At this point if you go back to CircleCI and try to run the build again CircleCI will return a fatal error: Unable to locate credentials. So why don’t we fix that next.

We need to first get the credentials from AWS and then provide them to CircleCI in order to allow the AWS cli to access and manage the S3 bucket. Best practice for this is to create a new Identity and Access Management (IAM) user specifically for CircleCI.

On the AWS console go to Security, Identity & Compliance and press IAM and then Add user.

In the Add User window type in CircleCI for User name, I already have an IAM user named CircleCI setup, So for the purposes of this tutorial and to illustrate these steps I will be using CircleCI2. Make sure you check Programmatic access for Access type.

For permissions choose Attach existing policies directly, and under Policy name check ‘AdministratorAccess’ and then click Create policy. This will provide your IAM user full access to your AWS S3 bucket.

After creating the IAM user, make sure you keep both the Access key ID and the Secret access key, as we will need them in the next step.

Now back to CircleCI, click on the settings button next to your project name to reveal the project settings menu then click on AWS Permissions. This is where you will paste the ID and Key from the previous step and then click “Save AWS keys.”

Now our CircleCI Container has both the AWS Command Line Interface tool and the credentials to access the AWS S3 bucket. The following steps will show you how to reveal your static site to the world.

In the AWS console go to Storage and then click on S3 and then click on the bucket we created earlier in this tutorial.

You will notice that the code from the repo has already been successfully deployed.

Now before you can access this static site you need to configure your S3 bucket for website hosting.

On the same screen click on Properties and then on Static website hosting.

In the following screen select Use this bucket to host a website and make sure you type in index.html for Index document.

By the way, the HTTP address provided above is your access endpoint. But if you try it in the browser unfortunately it won’t work and you will get an access denied error message. But that’s normal you still have one step to do: Set up your bucket policy.

This Bucket policy will allow access to the AWS S3 bucket to anyone via a browser.

You can read up here on bucket policies and examples if you want to learn more.

Now you can copy the code snippet above and paste it in your Bucket policy Editor and Voila!

If you see the screen above, then Congratulations! You have successfully set up Continuous Deployment to an AWS S3 bucket using CircleCI.

Now every time you push changes to your Github repo, CircleCI will automatically deploy the changes to your AWS S3 bucket.

You might have noticed that even though the deployment was successful CircleCI shows you a red NO TESTS warning.

This is normal because in a Test Driven Development (TDD) environment you would write tests first, and then before going to production your code needs to pass all the tests. An example with tests is beyond the scope of this tutorial, but suffice it to say that had we written tests, CircleCI would only have deployed if all our tests passed.

Using your own domain name to access this static site is also beyond the scope of this tutorial, but feel free to look here for instructions on how to configure Amazon Route 53 to route Internet traffic to your new site.

I am hoping to do a tutorial involving a Continuous Integration/Deployment example with a full battery of tests sometimes in the future. Meanwhile, if you have a moment, answer a short survey about this tutorial here, like it on LinkedIn or post a comment in the comments section.

Thanks for reading!