These devices inevitably use power and have batteries and circuits, which degrade over time. As time passes, these devices get replaced by newer upgraded ones. But this also means that more and more electronic waste builds up on our planet.

Also, since we have more electronic devices entering our ecosystem every day, we’re also using more power, leading to more carbon emissions.

This means we need to design both hardware and software in a way which acknowledges that the earth’s resources are limited. This article will give you an introduction on how you can architect your software for sustainability.

Here is what we'll cover:

• Prerequisites
• Why are we thinking about this now?
• What's involved in architecting and programming for sustainability?
• Profile your software for power and energy consumption
  • Energy Delay Product
  • Greenup, Powerup and Speedup
  • Tools for Profiling
• Software Techniques and Design Patterns for Sustainability
  • Consider which programming language to use
  • Make software Power aware: React to Thermal events
  • When appropriate use low power states on hardware
  • Investigate the tradeoffs between Interrupt driven I/O and Polling
  • Look into Caching
  • Design software with lifetime and updatability in mind
• Conclusion

##Prerequisites

I wrote this article with a focus on general ideas in computer science. This lecture will help with some background on Power and Energy in electrical circuits.

Other parts of this article provide references wherever you might need some background.

Why Are We Thinking About this Now?

Human made climate change has been steadily accelerating for decades. Carbon emissions have been rising. As a part of the technology community, we can help reduce some factors which have been contributing towards the degradation of the earth’s environment.

In this article I will present a few ideas and techniques on how you as a programmer and a software architect can make choices which will lead to more sustainable technology.

What's Involved in Architecting and Programming for Sustainability?

In the context of this article, when I describe architecting software and programming for sustainability, I’m referring to the following aims:

Designing software so that it:

1. Consumes the smallest amount of energy possible, to get the task at hand done.
2. Results in the least amount of battery degradation possible, while maintaining the minimum amount of performance needed for the task.
3. Needs a minimum amount of cooling for the hardware that it is running on.
4. Results in devices lasting longer.

You’ll see that the techniques we'll discuss here all have the same theme behind them: make electronic hardware do the minimum amount of work possible to complete the task at hand.

Let's start with some techniques that you can use to accomplish this while architecting and writing software.

##Profile Your Software for Power and Energy Consumption

A problem which cannot be measured, cannot be solved. As a result, measuring the energy and power consumption of software is one of the most important tasks we need to do when designing with sustainability in mind.

Let’s take a step back and cover some of the basics of power and energy.

As you may know, power is the rate at which energy is consumed. That is:

$$P = dE/dt$$

In continuous time domain, we could also say that:

$$Energy = \int_{0}^{t} P \,dt$$

Practically, we never really get power measurements in the continous time domain. We usually have discrete power measurements over an interval of time. So our Power vs time graph might look something like this:

Figure 1: Power vs time and computing energy consumption

Since we have n discrete Power measurements over time, the energy consumption can be estimated as the area under the curve. This can be modeled as the sum of n-1 trapezoids.

(Energy = \sum_{i=1}^{n-1} A^{i}) where (A^{i}) is the area of the ith trapezoid.

Profiling software for energy consumption refers to the process of measuring the energy consumed by the software being run.

So should you be profiling power or energy ? The answer is – it depends. In some scenarios which involve heavy user interaction (where the user controls the amount of time spent), it makes more sense to measure power.

For example: When a user is watching a video on a laptop or a mobile phone, they control many things such as the duration of the video watched, the screen’s brigthness, speaker volume, and so on.

In other scenarios where the user doesn’t have as much control over the work being performed by the electronic device, it makes sense for energy to be profiled. An example of such a case might be the energy needed to send a message via SMS / IP messaging service (not counting the energy needed to type out the message).

Let's look at some common metrics for profiling energy consumption next.

###Energy Delay Product (EDT)

Used in the research paper What are your programming language's Energy delay implications?, the energy delay product is a weighted metric which is defined as follows:

$$EDT = E * T^{w}$$

where E is the Energy consumption for the task and T is the time taken for the task to be completed.

This metric aims to give us a measure of how much energy an operation takes, while at the same time penalizing the metric for the amount of time taken. That is, an operation consuming the same amount of energy but taking longer to perform by a system, has a higher EDT and is therefore considered less energy efficient.

The weight w can be chosen to be:

• 1: When energy efficiency is of major concern
• 2: When both energy and performance are important
• 3: When performance is more important than energy efficiency

So, you can tailor w to give you an idea of your software system’s energy and performance, based on indicators fit for your use case – whether energy is more important or performance.

Greenup, Powerup, and SpeedUp

In the thesis Using the Greenup, Powerup And Speedup Metrics To Evaluate Software Energy Efficiency, AbdulSalam et al introduced new metrics to measure energy efficiency.

They saw that EDT had one shortcoming: since it was a product of two quantities (Energy and weighted time taken to complete the task at hand), it was possible that two systems which had the same EDT for a task, actually differed when it came to energy efficiency and performance - but their EDT was still the same.

As a result, it was hard to conclude which system was better from an energy + performance point of view, where energy and performance had equal importance.

They introduced 3 metrics to address this issue.

1. Speedup, which is defined as:

$$Speedup = T_{base} / T_{opt}$$

where (T{base}) = Time taken to complete the task for the unoptimized case, (T{opt}) = Time to complete the task for the optimized case.

If the optimized case is more performant than the unoptimized case (since remember, we could be optimizing for only energy, only performance, or both) then Speedup > 1.

2. Greenup, which is defined as:

   $$Greenup = Energy_{base} / Energy_{opt} = P_{base} * T_{base} / P_{opt} * T_{opt}$$

   .

Here (P{base}) is the average power consumed by the task in the unoptimized case and similarly, (P{opt}) is the average power consumed by the task in the unoptimized case.

If we look at different values of Speedup and Powerup, they can fall into the following categories:

1. Zone 1: Powerup < 1 and Speedup < 1 and Speedup > Powerup – in this scenario, the optimized solution sacrificed some performance, but there was a greater power reduction. As a result Greenup > 1, so there are energy savings.
2. Zone 2: Powerup < 1, Speedup > 1 – in this scenario, the optimized solution improved in performance and at the same time, reduced power consumption. As a result, energy consumption decreased and performance improved. This is the best case scenario for any optimization.
3. Zone 3: Powerup > 1, Speedup > 1 and Speedup > Powerup – in this case, average power consumption improved, but the speedup more than made up for the increase in power consumption. As a result, energy consumed still reduced between the optimized and unoptimized solutions.
4. Zone 4: Powerup > 1, Speedup > 1 and Powerup > Speedup – in this case, power consumption decreased, but so did performance. Still, the energy consumption as a whole increased since the loss of performance was greater than the power savings.
5. Zone 5: Powerup > 1, Speedup < 1 – in this case, the energy consumption increased since there was a performance degradation and also there was an increase in power consumption.
6. Zone 6: Powerup < 1, Speedup < 1, Powerup > Speedup – in this case, performance improved, but power increased more than the amount that performance improved. As a result, the energy consumption increased as a whole.

The figure below (inspired by this thesis) shows the zones where energy consumption increases and where it decreases. The red (4, 5 and 6) and the green areas (1, 2 and 3) depict these zones.

Figure 2 : The different zones for Speedup and Powerup

We went over a couple of metrics which both have their strengths and weaknesses. EDT’s strength is that you can give more weightage to performance. But its weakness is that many a time, it may not tell you whether one solution is really better than the other.

While knowing about energy consumption is very important, you should not ignore power consumption. High power consumption, amongst other effects, can have adverse thermal impacts on electronic devices. That is, your device’s skin and component temperature can increase. This may lead to:

1. Component damage: Electronic components such as memory, wires, and capacitors on your device can get damaged as temperature increases.
2. Battery damage: Batteries can get damaged at higher temperatures. Their life may decrease and in some extreme cases they may also explode.

Commercial electronics are usually built to operate within a 0 - 70 degrees Celsius temperature range. But for mobile phones it is usually recommended that the operating temperature not exceed 35 degrees celsius.

Before you profile the energy and power being consumed by a specific task on your devices, it's important to isolate the task as much as possible.

By this, I mean that its important that we take out as many other power consuming tasks or variables. These may be other services / daemons or settings on the device which may affect the power consumed in unreliable ways.

For example, if the device uses the internet, background processes may be doing some activities which may vary between multiple profile runs, so in general it is a good idea to switch off the internet (unless the task being measured uses the internet). We want to do this since we want reproducible power profiling numbers from run to run.

So in general, it is a good idea to make a checklist of the conditions which need to be constant for power profiling to be reliable and reproducible. This could vary from system to system.

##Tools for Profiling

Now, let's talk about how power can be profiled. Different systems may offer different ways of profiling energy consumed.

Depending on the operating system and the underlying hardware, there are a few options. I won’t go into details of these tools since there are many options and there is also good documentation available on their official websites.

###Linux

1. PowerStat is a tool that can measure power consumption on Intel hardware that supports the RAPL (Running average power limit) interface.
2. Cpu-energy-meter is a tool that measures the energy consumed by CPUs in a given time period.
3. Powertop is a tool authored by Intel which gives you information about many things of interest such as power consumption, CPU C / P States, CPU usage, and file system operations per second (and so on).

###macOS

1. Intel power gadget is a tool authored by Intel which gives users the ability to monitor power consumption, CPU frequency, CPU utilization, and even temperature on Intel based Mac machines. Note: According to Intel’s website, Power gadget will not get updates anymore and it recommends using Intel Performance Counter instead.
2. MxPower Gadget is a tool similar to the Intel power gadget, but for Apple silicon-based Macs.
3. Powermetrics is a command line tool that is pre-installed on Apple Silicon and Intel-based machines. It helps you obtain power measurements for CPU and GPU operation.
4. The Activity Monitor App is pre-installed on Macs, and gives you an overview of how your app is performing. It also gives you an overview of various metrics about your Mac. While it doesn’t give you power numbers, it is still a useful tool.

###Android

The AOSP (Android Open Source Project) official documentation provides some guidelines on how to measure system component power here. This process may vary depending on the manufacturer of the device.

Android Studio, Android’s official development IDE also offers a Power Profiler to measure power. The On-Device Power Monitor (ODPM) reports power consumed by all profilable sub-systems through their power rails. Note: These measurements are not specific to any app, as they measure device power on the whole.

###iOS

During iOS app development, developers can measure the energy impact of their app using XCode’s inbuilt Instruments profiler.

We talked about various ways to profile power and energy in this section. In the coming sections, I'll discuss some techniques and considerations to keep in mind to make your software more sustainable.

##Software Techniques and Design Patterns for Sustainability

###Consider which programming language to use

Programming languages have multiple levels of abstraction over the hardware that they are running on. Some have a model of the machine presented to the programmer which is really close to the actual hardware they are running on.

Languages which are directly compiled to machine code are generally more power efficient. They also are generally more performant, since there is less overhead compared to languages which are interpreted or run on a virtual machine, such as Java.

The research paper I mentioned earlier (What are your programming language's Energy delay implications) discusses this in detail and is a great read.

While many popular operating systems have the majority of their APIs in languages like Java, Kotlin, and Swift (which are not native), they usually have performance critical APIs available in native code as well – for example, Android’s C interface-based NDK library. These can interface with their virtual machine-run counterparts and in combination be used to write efficient applications.

###Make software power aware: React to thermal events

In the previous section, I talked about the importance of keeping power consumption low on the devices which run your software. This is not only important to reduce energy consumption, it's also important so that your device’s operating temperature doesn’t go into territory which can damage your device.

The operating temperature of your device doesn’t only depend on the software running on it (which includes your software). It also depends on the ambient temperature.

For example: if a smartphone is used in a car on a hot day, there’s a good chance that when a demanding application is run on it, it’ll thermally stress out the device. So what can you as a software architect do about this?

You cannot control ambient temperature, but you can control how your software responds to thermal events. Many popular operating systems offer APIs to ‘listen’ for thermal events. When apps get notifications that the device’s thermal load has crossed a certain limit, they should take appropriate actions.

For example, if a device is streaming video and it gets a notification about thermally heating up, it may lower the resolution at which it is streaming video in order to reduce power consumption.

If a video conferencing app gets a worrisome thermal event notification, it may want to reduce the frame rate at which it is capturing video / reduce the resolution.

The following are thermal event APIs on popular operating systems

MacOS and iOS: the NSNotificationCenter offers a thermal event listener which userspace applications can register for.

Android: PowerManager offers an onThermalStatusChangedListener which apps can register for, to receive thermal event notification.

###When appropriate, use low power states on hardware

Most modern processors have provisions for reducing power consumption. In these modes, the processor consumes less power by changing the CPU clock speed, putting the CPU into various idle states among various methods. Operating systems give the user the ability to leverage these low power states.

For example, Linux has the cpupower-frequency-set tool which allows users to modify CPU frequency settings.

As another example, Arduino microcontrollers give programmers the ability to put them in various sleep modes when there are no active tasks to be run. These microcontrollers are typically woken up by an interrupt when a task to be processed is ready.

###Investigate the tradeoffs between Interrupt driven-I/O and Polling

Often, software needs to wait on another component of hardware or software for data to process. There are usually two parties involved: the producer of the data and the consumer of the data

For example, a mobile application may be waiting on a touch event on the display. Here the consumer is the mobile application and the producer is the display hardware + software.

Another example could be a native embedded program that is waiting on the status of a CPU register to change in order to perform some task. Here the consumer is the native embedded program and the producer is the CPU register (well, the producer may actually be producing some other data, but for simplicity we’re only considering a CPU register here).

There are broadly 2 ways of doing this:

Polling: The process of checking the status of a data-readiness at regular predetermined intervals of time is referred to as polling. Typically it looks something like this in code:

// psuedo-code
while(poll) {
    bool data_available =  check_data();
    if (data_available) {
        process_data();
    }
    sleep(SLEEP_TIME); // to avoid wasting cpu cycles
}

Polling seems pretty simple to implement. But unless you’re absolutely sure that the data or condition you’re waiting on is going to be available at a regular frequency, it has some disadvantages:

1. Polling doesn’t respond to the data being available as soon as it is available, since it checks for availability after every SLEEP_TIME ms.
2. By constantly checking every time the polling thread wakes up, it still wastes cpu cycles and as a result some power.

Interrupt-driven I/O: in this strategy, there are no explicit checks for the data that is being waited on to be ready. Instead, it is the responsibility of the producer to inform the consumer whenever the data is ready. The consumer does not explicitly check in with the producer, even periodically. As a result, this saves CPU cycles and also power!

There are multiple ways of implementing interrupt-driven I/O. There can be hardware interrupts as well as software interrupts.

For example, when a user touches a mobile phone screen at a particular location, the operating system could send a callback to one of the application’s thread’s notifying it that a touch input event is ready to process. Until the thread gets the notification, it can either do something else, or just stay asleep!

So in short, interrupt-driven I/O is generally more power efficient than polling, and you should prefer it, unless there’s a very good reason to use polling.

Look into Caching

Caching in computer science refers to the process of saving data in a storage location that is typically faster to access than high latency storage (which are generally larger in capacity).

Caching is useful when the same set of data needs to be read or modified repeatedly by software. It allows efficient retrieval and re-use of data.

You can see caching in action in many forms such as:

1. Random Access Memory (RAM) is a cache for underlying disk
2. CPU cache – L(n) – where n is the level of the cache. As n decreases, the cache size usually gets smaller and the latency of data retrieval gets smaller as well.
3. Applications can cache data retrieved from the internet into their on device memory for quick access.
4. Local servers can cache data from remote servers for quicker access as well.

And many more.

Figure 3: Flow of data from disk to CPU via cache hierarchy

As a result of being more efficient with data access, caching usually reduces power consumption and saves energy as well.

Before software can take advantage of caching, it needs to be architected so that that's possible. In general, the following steps can help you:

1. The very first thing is to make sure you're on the lookout for opportunities to cache data.
2. Then, implement caching. How you implement it will depend on the type of software you’re building. It might range from tiling data matrix operations to caching data retrieved from the internet for re-use later on.
3. Monitor profile performance, energy and power usage – and make sure that these metrics are actually improving. It is possible with sub-optimal implementations to cause thrashing of the cache – which may increase power consumption and reduce performance.

###Design software with lifetime and updatability in mind

This article has talked a lot about your software’s energy and power consumption. Finally, there’s something else that is very important as well: the ability for your software to last a long time and be updatable.

It's very beneficial for software to last a long time, since that in turn means that devices will work well for longer. Software being updatable means that older devices get important feature and security updates, which leads to users not necessarily needing to buy newer devices just to get better software. This leads to a reduction in electronic waste and energy being spent on recycling electronic components.

Software updatability is a large topic which I will not cover in detail here. But there are some principles that you shoulder consider which will serve you well while designing for updatability:

Design your software in modules which are updatable by themselves: This may involve having strict interfaces between different modules so that on updating one module the software package as a whole still functions correctly.

Focus on using memory as efficiently as possible: If all software was designed keeping in mind memory efficiency, electronic devices would last longer, since the amount of memory they had would be sufficient for longer.

As an example, consider the iPhone: the first generation in 2007 had 128MB of RAM and 16GB of flash storage (maximum). Today, the iPhone 15 has 6GB of RAM and has a maximum storage option of 1 TB (1024GB). That is a nearly 16x increase in RAM and 64x increase in flash memory. This was needed since the amount of memory needed by applications and the operating system itself grew in size by a huge amount.

As time passes, if memory is used judiciously, we can think of a future where electronic devices last for multiple decades instead of being phased out every 5-6 years.

Testing needs to be bulletproof: When software is going to be updated frequently, there’ll be multiple version of software running on devices which interact with other pieces of software which might be older. These pieces of software must still function correctly.

For this to happen, before deploying updates, software must be tested thoroughly. As an example: when an app developer releases an app, they must test with multiple versions of operating systems to make sure that their app behaves well on all of them.

##Conclusion

This article introduced the concept of engineering software for sustainability and discussed why this is necessary.

You also learned about various metrics as well as some techniques to make your software sustainable and more efficient.

Finally, I hope it inspired you to think about a very serious problem we're facing right now and how we in the tech community can do our part.

I hope you enjoyed the article!

This improves program performance since operations are no longer being computed on the fly, during runtime.

While these techniques of offloading operations to compile time improve performance, they can also help subtle problems such as a the 'Static initialization Order Fiasco' which I covered in a previous article.

This tutorial teaches you two ways to make this happen in C++. Here is what we'll cover:

• Prerequisites
• How to evaluate functions at compile time using constexpr
• The constinit specifier and its uses
• Summary

##Prerequisites

• A basic understanding of C++: For readers not familiar with C++, Learn C++ Programming for Beginners – Free 31-Hour Course is a helpful resource
• A read through of my previous article What is the Static Initialization Order Fiasco in C++ will benefit you with context around constinit.

How to Evaluate Functions at Compile Time Using constexpr

To understand this, let's first take a look at an example of a function which performs a very simple computation:

int add2(int input) {
    return input + 2;
}

int main() {
    int b = add2(3);
    std::cout << "b = " << b;
    return 0;
}

Here, all that the function add2() does is it adds 2 to the input. In main(), add2() was called with input 2. So, it's pretty straightforward for anyone looking at the program to tell what the output of the function is going to be: 5. There is no sort of non-determinism here.

But if we look at the x86 assembly code generated by the compiler it would look similar to the below:

add2(int):
        push    rbp
        mov     rbp, rsp
        mov     DWORD PTR [rbp-4], edi
        mov     eax, DWORD PTR [rbp-4]
        add     eax, 2
        pop     rbp
        ret
.LC0:
        .string "b = "
main:
        push    rbp
        mov     rbp, rsp
        sub     rsp, 16
        mov     edi, 3
        // Function add2() getting called
        call    add2(int)
        mov     DWORD PTR [rbp-4], eax
        mov     esi, OFFSET FLAT:.LC0
        mov     edi, OFFSET FLAT:_ZSt4cout
        call    std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)
        mov     rdx, rax
        mov     eax, DWORD PTR [rbp-4]
        mov     esi, eax
        mov     rdi, rdx
        call    std::basic_ostream<char, std::char_traits<char> >::operator<<(int)
        mov     eax, 0
        leave
        ret

(Note that the reference assembly codes in this article were generated by using the compiler explorer tool at godbolt.org.)

The function add2() actually gets called at runtime in main() with the call add2(int) line.

Since this function does something that can be completely computed before the program actually executes (2+3 = 5, even in our heads!), wouldn't it be cool if we could have the compiler not create a function for this operation and just fill in the answer directly in assembly code? This is exactly what constexpr does.

If the code was changed to this:

constexpr int add2(int input) {
    return input +2;
}

int main() {
    int b = add2(3);
    std::cout << "b = " << b;
    return 0;
}

the compiler would output the following assembly code:

.LC0:
        .string "b = "
main:
        push    rbp
        mov     rbp, rsp
        sub     rsp, 16
        mov     DWORD PTR [rbp-4], 5
        mov     esi, OFFSET FLAT:.LC0
        mov     edi, OFFSET FLAT:_ZSt4cout
        call    std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)
        mov     rdx, rax
        mov     eax, DWORD PTR [rbp-4]
        mov     esi, eax
        mov     rdi, rdx
        call    std::basic_ostream<char, std::char_traits<char> >::operator<<(int)
        mov     eax, 0
        leave
        ret

The function add2() has disappeared and the line mov DWORD PTR [rbp-4], 5 has baked into the program the evaluation of the function add2() at compile time. There is no runtime call to add2().

Mind you, this is possible since we've passed in 3 – an expression that is known at compile time – to the add2() function. If something that couldn't be evaluated at compile time was passed in, the compiler would again generate an add2() function.

You can see what I mean in this snippet:

#include<iostream>
#include <random>

constexpr int add2(int input) {
    return input +2;
}

int main() {
    int rd = std::rand();
    int b = add2(rd);
    std::cout << "b = " << b;
    return 0;
}

The assembly generated again has the add2() function:

add2(int):
        push    rbp
        mov     rbp, rsp
        mov     DWORD PTR [rbp-4], edi
        mov     eax, DWORD PTR [rbp-4]
        add     eax, 2
        pop     rbp
        ret
.LC0:
        .string "b = "
main:
        push    rbp
// Snip

Alright, you've now seen how using the constexpr specifier can help programs move runtime costs to compile time costs in many cases.

Let's now look at another specifier introduced recently, in C++20, which verifies that variables are initialized at compile time.

The constinit Specifier and its Uses

The constinit specifier was introduced in C++ 20. This specifier asserts that a variable has constant initialization – it sets the initial values of the static variables to a compile-time constant. Otherwise, the program is ill-formed and the compiler produces an error. For example:

int add2(int v) {
    return v + 2;
}

//Error: 'constinit' variable 'glob' does not have a constant initializer  
constinit int glob = add2(2);

int main() {

    return 0;
}

The compiler errors out here since the add2() function isn't sure to have constant initialization – which can be determined at compile time. Now if the add2() function is marked constexpr, it will have constant initialization, so the code compiles.

constexpr int add2(int v) {
    return v + 2;
}

//OKAY 'constinit' variable 'glob' does have a constant initializer.  
constinit int glob = add2(2);

int main() {

    return 0;
}

Now you may ask – what's really the use of this specifier?

The answer is it that can be used in certain cases to solve the 'Static Initialization Order Fiasco'.

I talked about the 'Static Initialization order Fiasco' in an earlier article. If we use constinit, the compiler is giving the programmer its word that the constinit variable will be constant initialized – so that's before any other static variables are constructed at runtime. We get rid of the 'Static Initialization / Destruction Order Fiasco'.

Another example where we use strings that are constant initialized illustrates this:

// Parent.h
#pragma once
class Parent {
    public:
       size_t getMoneyCount();
       constexpr Parent(const char *moneyString): mData(moneyString) {};
    private:
        std::string_view mData;     
};
extern Parent everyonesParent;

// Parent.cpp
#include<Producer.h>

constinit static Parent everyonesParent("TheParent");

size_t Parent::getMoneyCount() {
    return mData.size();
}

//Child.cpp
#include<Child.h>

class Child {
    public:
    Child(Parent &parent) : mMoneyCount(parent.getMoneyCount()) {};
    private:
    size_t mMoneyCount;
};

static Child everyonesChild(everyonesParent);

There is no static initialization order problem here, since the static object everyonesParent is guaranteed to be initialized before everyonesChild, since it was marked constinit.

It was okay to mark everyonesParent constinit since it used std::string_view which can be constant initialized – unlike std::string. Also, it had a constexpr constructor. If it didn't use either of these, compilation would have failed!

In closing, something to note about constinit: constinit does not imply const.

constinit values can be modified after construction. Take this example – it is perfectly legal:

#include<iostream>

constinit int i = 42;
int main() {
 i++;
 std::cout << " i is " << i << "\n";
}

##Summary

This article covered compile time operations and run time operations. It analyzed how compilers produce code to either generate functions used at runtime or evaluate them at compile time.

You learned about the constexpr and constinit specifiers in C++, and how they are extremely useful.

I hope you enjoyed the article!

I'll first go over what the problem is, then go onto some solutions and explore how they work. Let's get started.

This is what we'll cover:

• Prerequisites
• What is the 'Static Initialization Order Fiasco' in C++ ?
• Solutions to the static de-initialization order problem
• Construct on first use idiom
• Nifty Counter Solution
• Summary

##Prerequisites

• A basic understanding of C++: For readers not familiar with C++, Learn C++ Programming for Beginners – Free 31-Hour Course is a helpful resource
• In particular, an understanding of storage classes in C++ will be helpful.

##What is the 'Static Initialization Order Fiasco' in C++ ?

The C++ standard states:

"The order in which static objects are initialized across different translation units is undefined or ambiguous."

A translation unit is just a way of saying a file that is fed into the compiler. It's a C++ source file with all the code from the headers included in it.

One thing to note though, for later in the article: static objects in the same translation unit are constructed in order of declaration and destructed in the reverse order.

So, how is this a problem?

It can be a problem in the following situation:

Let's say there are 2 static objects in 2 different files. File1.cpp has a static object of type class A – aObj . File2.cpp has a static object of type class B – bObj. The static object in File1.cpp is visible to File2.cpp since it declares aObj as extern in File1.h.

// Static initialization order problem
// File1.h
class A {
....
  void doSomething() {
    ...
  } 
}
extern A aObj;

//File1.cpp


static A aObj;

// File2.cpp

class B {
B() {
 aObj.doSomething();// Not okay! aObj may not have been constructed
}
....
}

static B bObj;

In this program, it is possible that the object aObj in File1.cpp gets initialized before bObj in File2.cpp. That is all good since in that case, the constructor for bObj runs after aObj has been constructed. It is safe to call call methods on aObj.

But it is also possible that the object bObj in File2.cpp gets initialized before aObj in File1.cpp. In that case, the constructor of bObj calls doSomething() on aObj which has not been constructed! The memory has been allocated for aObj, but it hasn't been constructed. This could lead to unintended behavior / a corrupt program.

So this is what the static initialization order fiasco is all about.

But we're not done: the other problem is the static de-initialization order fiasco! This is pretty much the same problem, just applied to the order of de-initialization of static objects.

The C++ standard doesn't specify the order in which static objects get de-initialized as well. So it is possible that static object aObj gets destroyed before bObj. This is a problem if bObj's destructor uses or references aObj.

This is illustrated in the code snippet below – it is pretty much the same as the example above, just that its the de-initialization order which is dangerous this time:

// Static de-initialization order problem
// File1.h
class A {
....
  void doSomething() {
    ...
  } 
}
extern A aObj;

//File1.cpp

static A aObj;

// File2.cpp

class B {
B() {}
~B() {
 aObj.doSomething(); // Not okay! aObj may have already been destructed!
}
....
}

static B bObj;

Note: These problems are only applicable to objects with static storage scope. They won't occur if bObj was a variable with automatic storage scope. In that case, the C++ standard guarantees that aObj is constructed before bObj and destructed after it.

Another note: These problems also do not occur in C programs. Why is that so? Well in C, there's no concept of constructors and destructors. Static objects are completely defined during compile time.

How to Solve the Static De-initialization Order Problem

Now that it is clear what the problem is, I will discuss some solutions. There are multiple ways of solving this problem – each with its tradeoffs. Let's take a look.

###Construct on first use idiom:

This idiom tries to make sure that there is always a fully constructed object whenever the static object in question is used. Following the examples in the previous section, we can do this by replacing all references to aObj by a function call aObj() which returns a reference to an object of type A.

In code it looks like this:

// Static initialization order problem
// File1.h
class A {
....
  void doSomething() {
    ...
  } 
};

A& aObj();

//File1.cpp

A& aObj() {
  static A *aObj = new A();
  return *aObj; 
}

// File2.cpp

class B {
 B() {
   /*
    * Okay since calling aObj() gaurantees that
    * static A *aObj = new A(); ran
    */
   aObj().doSomething();  
  }
  ....
};

static B bObj;

bObj can safely assume that calling aObj() returns a fully constructed aObj since this line:

static A *aObj = new A();

would have run on the function call and will give it a fully constructed object. Also, since the program never calls delete on aObj, it is never destructed so it is also safe to use aObj in bObj's destructor.

But this does mean that the memory allocated for aObj always stays alive and valid throughout the lifetime of the program. And this may or may not be a problem (it does get reclaimed by the OS after the program exits, of course).

So, in which situation is this solution not great? In the case that aObj's destructor does something desirable. For example: when aObj gets destructed – it writes to a log file / does something else that has some side effects.

Now you may ask, okay, why don't just I replace the static pointer in the aObj() function call with a static aObj object?

A& aObj() {
  static A aObj;
  return aObj; 
}

That still ensures that aObj has been fully constructed by the time the function is called right? Right. But it does not save us from the static de-initialization order problem. It is still possible that aObj's destructor runs before bObj 's destructor.

There is an interesting trick that solves both of these problems: The Nifty Counter Idiom.

###Nifty Counter Solution

Reference: this resource on the Nifty counter idiom presents the idea behind this idiom. Let's examine it.

The idea is to ensure that:

1. The static object being used gets constructed before any other static object in the translation unit that it is being used in.
2. The static object being used gets destructed after any other static object in the translation unit that it is being used in.

// File1.h
#pragma once

struct A {
  A();
  ~A();
};
extern A& aObj;

static struct AInitializer {
  AInitializer ();
  ~AInitializer ();
} aInitializer; // static initializer for every translation unit that aObj is used in

// File1.cpp
#include "File1.h"

#include <new>         // Used for placement new
#include <type_traits> // Used for aligned_storage

static int niftyCounter; // this is zero initialized at load time

/*
 * Memory for the static object aObj - memory itself is valid throughout the
 * the lifetime of the program.
 */
static typename std::aligned_storage<sizeof (A), alignof (A)>::type
  aObjBuf; 

A& aObj = reinterpret_cast<A&> (aObj);

A::A ()
{
  // Construct A
}
A::~A ()
{
  /*
   * Destruct A: with possible side effects
   * like writing to a file.
   */
} 

AInitializer::AInitializer ()
{
  if (niftyCounter++ == 0) {
    new (&aObj) A (); // use placement new operator
  }
}

AInitializer::~AInitializer ()
{
  if (--niftyCounter == 0) {
    (&aObj)->~A(); // run the destructor
  }
}

Let's try to understand what this code does.

First, in the header file, File1.h has the definition of class A first. After that, is have the definition of a class called AInitializer.

There is also a static object defined in the header file of type AInitializer. This makes sure that the constructor for AInitializer runs before the constructor for any other static object in the translation unit that File1.h is included in (of course you have to include File1.h before any other static object's definition in source files).

Remember: static objects in the same translation unit are constructed in order of declaration and destructed in the reverse order.

So now that AInitializer is constructed before any other static objects in a translation unit, how can we use this to our advantage? aObj can be constructed in the constructor of AInitializer! Which is what is happening in the lines below:

AInitializer::AInitializer ()
{
  if (nifty_counter++ == 0) {
    new (&aObj) A (); // use placement new
  }
}

Note that the placement new operator is being used here instead of the new operator to construct aObj. Let's see what would happen if we used new instead. The code would look like this:

A& aObj;
A *aObjp = nullptr;

AInitializer::AInitializer ()
{
  if (nifty_counter++ == 0) {
    aObjp = new A (); 
    aObj = *aObjp; // Not okay! Cannot re-assign a reference
  }
}

This doesn't work since a reference needs to be defined and declared at the same time. That is precisely why the placement new operator needs to be used.

static typename std::aligned_storage<sizeof (A), alignof (A)>::type
  aObjBuf; 

A& aObj = reinterpret_cast<A&> (aObj)

This allocates memory to fit an object of type A and later assigns that to the reference. Now all that's left to be done is to actually construct the object in AInitializer's constructor – which is what is done with the placement new operator.

Another question that may arise in your mind: here, there is a static object aObjBuf. But isn't that subject to the same de-initialization order problem that we talked about in the second part of the Construct on first use idiom?

The answer is that the memory for aObjBuf stays alive and valid until the program is alive. Nothing happens in the construction of the memory. So it's valid to this.

This approach also makes sure that the static de-initialization order problem isn't hit, since the last AInitializer object destructed will call the destructor of aObj. That is guaranteed to run after any static objects in other translation units run, since within the particular translation unit, the static object aInitializer is declared before any other static object using aObj. This means it will get destructed in the reverse order – that is after the destructor for any other static objects have run.

There are some caveats here: this solution isn't the easiest to understand and implement. This is also not thread safe. You can find more information in the article on Nifty counters presented in the The C/C++ Users Journal, May, 1999 here.

##Summary

Using statically initialized objects in C++ is tricky and should be done with care. Fortunately, there are multiple solutions and ways to get around the problem.

In this article, we covered some common solutions: the 'Construct on first use' idiom and the 'Nifty counter solution', along with their merits and challenges.

Hope you enjoyed this article!

• Measure/profile the performance of certain parts of code.
• Do work at certain periods of time, from within a program.
• Detect whether threads are in a deadlock / taking too long to complete an operation.
• Synchronize tasks between different components of software

and many more…

This article will guide you through how you can measure time in modern C++.

Prerequisites

• A basic understanding of C++: For readers not familiar with C++, Learn C++ Programming for Beginners – Free 31-Hour Course is a helpful resource.
• A quick read through Linux time tracking infrastructure – such as you can find here – will help you get familiar with the ideas presented in the article.

Common Ways to Track Time in C++

This article covers how you can keep track of time in C++. In C, on UNIX like systems, you can use the clock_gettime() function to keep track of time. It returns time in a structured way through the timespec struct.

The clock_gettime() /gettimeofday function gives us back a filled timespec struct which has two fields:

1. tv_sec, which gives us the time in seconds since the time source – CLOCK_REALTIME / CLOCK_MONOTONIC that was passed into clock_gettime. The 'type' of this field is time_t which is usually an integral value.
2. tv_nsec, which gives the time after tv_sec, in nanoseconds since the time source that was specified while calling clock_gettime(). The type of this field is a long int.

So why is clock_gettime() not good enough? The answer is that the members of struct timespec can easily be passed to functions as they're really just ints / floats. They're not strongly typed.

It's also easy to forget about the units in which they represent time while passing information around to functions. This can happen when you're dealing with projects that have thousands of lines of code.

So what's the solution?

##The std::chrono API

C++11 introduced the std::chrono API, which can help you avoid some of these problems.

There are 3 important parts of the API.

###1. std::chrono::duration

As its name suggests, std::chrono::duration is a type that represents a time interval. The official C++ reference mentions that std::chrono::duration is a templated type with the following signature:

template<
    class Rep,
    class Period = std::ratio<1>
> class duration;

Here, the Rep template parameter represents the type that is used to count 'ticks' of time. A tick is just a unit of time which is a given fraction of a second. Period, the second parameter, defines what exactly that fraction is.

So, for example, if you write:

using my_ms_type = std::chrono::duration<int, std::ratio<1, 1000>>

my_ms_type duration_ms duration = 3; // error: cannot convert from int
my_ms_type duration_ms duration_ok{3} // OK, can construct from int

my_ms_type is a type that has been defined, which counts in units of milliseconds (1/1000th of a second). This count is expressed as an integer. As you might be able to guess, the Rep template parameter is int and Period is std::ratio<1,1000> (which really is a way of saying 1/1000).

Now that it's clear how durations are represented, let's see what we can and cannot do with these.

If there is a function that takes in a my_ms_type duration and you instead try to pass in any non-std::chrono::duration type, you'll get a compiler error.

It is possible to implicitly convert between different types of std::chrono::duration as long as information isn't lost with the type of Rep, since the standard library can compute the relationship between two std::chrono::durationtypes. It is not possible to implicitly convert if there is a loss of information. For example:

#include<chrono>

using namespace std::chrono;
using my_type_ms = std::chrono::duration<int, std::ratio<1, 1000>>;
using my_type_ms_f = std::chrono::duration<float, std::ratio<1, 1000>>;
using my_type_hundredth_s = std::chrono::duration<int, std::ratio<1, 100>>;
void f(my_type_ms millis) {}
int main()
{
   int duration = 2;
   my_type_ms_f duration_f{2.5};
   my_type_hundredth_s duration_compatible{100};

   f(duration); // error: could not convert 'duration' from 'int' to 'my_type_ms'

   f(duration_f) //error: since float -> int will lose information

   f(duration_compatible) // OK since no information is lost
}

The standard library also has some predefined std::chrono::duration template specializations for common time durations such as std::chrono::duration::seconds, milliseconds, microseconds, and so on.

You can also get the 'count' value contained in a duration by using the count method in a duration.

std::chrono::seconds duration{3};
// Prints: 'Duration count: 3 seconds'
std::cout << "Duration count: " << duration.count() << " seconds";

Interestingly, converting from a unit with higher precision like nanosecond to something with a lower precision such as millisecond may also lead to a loss of information. For these specific cases, you need to use an explicit cast for conversion. This is called duration_cast. For example:

nanoseconds durationInNs = 3000000000;
seconds ms = duration_cast<seconds>(durationInNs); //OK 3s
durationInNs = 3500000000;
ms = duration_cast<nanoseconds>(durationInNs); // OK 3s - truncates down

Now that we know why std::chrono::duration is useful, let's move on. The next section explores std::chrono::time_point.

###2. std::chrono::time_point

std::chrono::time_point is a way of expressing a particular point in time – surprise, surprise!

If you think about it, how can you logically define a point in time ? We need to have a reference starting point and a duration from the starting point. This is exactly what std::chrono::time_point does.

The class declaration looks like this:

template<
    class Clock,
    class Duration = typename Clock::duration
> class time_point;

There are two template parameters here:

The first one is Clock which represents a reference clock relative to which the point in time is being measured. For now, some examples of clocks are:

• system_clock: this represents a real-world wall clock. It's useful when you want to measure time in terms of real-world times. It's important to note that the system time can usually be changed on any system, so you shouldn't depend on this clock to calculate time periods between tasks / performance profiling.
• steady_clock: this represents a monotonically increasing clock. It's useful when you need stop-watch like clock accounting.

The second template parameter is Duration which is what we discussed in the previous section. A time_point needs to be associated with a duration type since that's what is being used to measure ticks since the 'epoch' of the Clock.

Epoch is just a way of saying a reference point in time. While there's no mandate for which reference to use, Unix Time - that is, the time since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970 is a common one.

Time points based on the same clock can be subtracted and not added. For example:

auto tp1 = std::chrono::system_clock::now();
...
auto tp2 = std::chrono::system_clock::now()
auto tp3 = std::chrono::steady_clock::now();

auto diff = tp2 - tp1; // OK
auto add = tp1 + tp2; // Not Ok
auto add = tp3 - tp2; // Not Ok - based on different clocks

Let's now see what clocks are.

###3. Clock Types

A Clock is a type that ties together std::chrono::duration and std::chrono::time_point. It has a function now() that returns the current time_point. The formal requirements for a type to be a Clock can be found in the C++ spec here.

As mentioned before, system_clock and steady_clock are two popular clocks provided by the standard library. Each clock has its own associated duration as well.

Each time_point is associated with some clock, since it really has to be relative to some given reference.

Finally, let's see some examples of how you can tie together duration, time_point, and Clock. Let's say you want to measure the time that looping 100,000,000 times takes in nanoseconds, and you also want to print out the current wall time:

#include <chrono>
#include <iostream>
#include <ratio>
#include <thread>
#include <ctime>

using namespace std::chrono;
constexpr size_t kIterations = 100000000;
void testFunction () {
    for (size_t i = 0; i < kIterations; i++) {
    }
}

int main()
{
    auto tStartSteady = std::chrono::steady_clock::now();
    std::time_t startWallTime = system_clock::to_time_t(system_clock::now());
    std::cout << "Time start = " << std::ctime(&startWallTime) << " \n";
    testFunction();
    auto tEndSteady = std::chrono::steady_clock::now();
    nanoseconds diff = tEndSteady - tStartSteady;
    std::time_t endWallTime = system_clock::to_time_t(system_clock::now());
    std::cout << "Time end = " << std::ctime(&endWallTime) << " \n";
    std::cout << "Time taken = " << diff.count() << " ns";
    return 0; 
}

The output of the program is the following:

Output:
// This can of course vary from system to system
Time start = Tue Nov  7 07:11:13 2023

Time end = Tue Nov  7 07:11:13 2023

Time taken = 50998885 ns

Summary

This article explored various facets of the std::chrono API in C++. The std::chrono API allows C++ programmers to safely keep track of time thanks to its strongly typed system. It also helps maintain support for convenient conversions between different 'types' of time points.

This article will show you how to use Clang's front end libraries to build a simple static analysis tool which will operate on C++ source / header files. It will use the power of AST (Abstract Syntax Tree) traversal.

An abstract syntax tree is a tree structure representing the syntactical structure of code. Here is a good explanation of how it works, and here is a tool to help you explore the AST for a given piece of code.

Here, I will teach you how you can use the Clang AST to find information about the code given to it to show you how powerful it is.

This article goes through everything step by step, and I’ll explain the terminology I'm using briefly at each step.

In the first section, you'll learn how to get the open source Clang project. Then, we'll explore how you can build a static analysis tool with a simple goal: to check if each Class defined in a source / header file starts with an uppercase character. We'll do this using Clang's frontend libraries which will analyze the C++ source AST.

So go ahead and grab your favorite coding beverage, get comfortable, and read on!

Here's what we'll cover:

• Prerequisites
• How to Get the Clang Project and Access the Front-end Libraries
• How to Create the Scaffolding for the Static Analysis Tool
• Putting it All Together in Code
• Summary

Prerequisites

Before getting started, it would be beneficial to have a basic understanding of the following:

• Compilers: this page is a good primer for beginners.
• C++ : For readers not familiar with C++, Learn C++ Programming for Beginners – Free 31-Hour Course is a helpful resource.
• Git: Git Best Practices – A Guide to Version Control for Beginners is an excellent starting point.

How to Get the Clang Project and Access the Front-end Libraries

Since clang and llvm are open source projects, they have very comprehensive documentation around how to get started with getting the code and building tools using them.

You can check out the Getting Started page of the llvm project to get more information about this. I've referenced that in this article as well.

###1. Get the Clang project

On a UNIX like terminal, clone the llvm Git project into your own directory. I'll call it ast-anaylzer.

1. mkdir -p ~/ast-analyzer; cd ~/ast-analyzer
2. git clone https://github.com/llvm/llvm-project.git #Clone the llvm project source

###2. Get the CMake build system and Ninja build tool

CMake and ninja work in conjunction to form a build system. CMake generates build.ninja files, which contain commands that tell ninja how to generate output targets. We’ll get into this more a little later.

####2.1 Get and install Ninja

Here are the steps you can follow to install Ninja:

1. cd ~/ast-analyzer
2. Clone the ninja source project with this command: git clone https://github.com/martine/ninja.git
3. cd ninja
4. Checkout the release branch - this is the stable branch - with this command: git checkout release
5. python3 configure.py –bootstrap This prepares and creates a Ninja binary (configure.py –help will give you more information).
6. Install ninja with this command: sudo cp ninja /usr/local/bin. After this step, as a basic validity check, do which ninja to make sure it says /usr/local/bin/ninja.

####2.2. Get and install CMake

Here are the steps you can follow to install cmake:

1. cd ~/ast-analyzer
2. Clone the cmake project source code: git clone git://cmake.org/stage/cmake.git
3. cd cmake
4. Checkout the release branch - this is the stable branch - with this command: git checkout release.
5. Run the bootstrap script: ./bootstrap. This prepares cmake to be built and installed on your host machine.
6. Build cmake from source with this command: make.
7. Then finally install cmake: sudo make install.

Once we’ve gotten Clang, we’ll build it and configure it so we can build Clang-based tools as well.

###3. Build Clang and configure it

Create a ‘build’ directory. This is where our build.ninja/ output binaries and so on will get created:

cd ~/ast-analyzer; mkdir -p build; cd build

Now we need to generate the build.ninja file in order to build Clang and also the tools in the directory from the project cloned earlier (llvm/clang-tools-extra). You can do this using CMake like this:

cmake -G Ninja ../llvm-project/llvm -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" # Enable the clang-tools projects in our build as well

This should generate a build.ninja file, which I encourage you to open and check out the contents. You will see that it contains a list of targets followed by dependencies. For example, one of the targets may look something like this:

#############################################
# Utility command for install-llvm-headers

build install-llvm-headers: phony CMakeFiles/install-llvm-headers llvm-headers

We’ll also do this for the custom static analysis tool we build in the next steps.

###4. Build and install all targets specified in the build.ninja file

ninja; ninja install

Okay, the setup is done and now we get to the fun part!

How to Create the Scaffolding for the Static Analysis Tool

We’ll be building our tool as a part of the clang-tools-extra directory in llvm-project/clang-tools-extra. Let's go ahead and create that directory. We’ll call our tool class-analyzer.

mkdir ~/ast-analyzer/llvm-project/clang-tools-extra/class-analyzer
cd ~/ast-analyzer/llvm-project/clang-tools-extra/class-analyzer

Now we need to create a CMakeLists.txt. This is basically a file that tells the CMake build system to add the source files in this tool to the build.ninja file it will generate. This lets ninja know how to build our tool.

Our CMakeLists.txt file will look like this:

CMakeLists.txt

set(LLVM_LINK_COMPONENTS support)
set(CMAKE_CXX_COMPILER /usr/bin/clang++)  


add_clang_executable(class-analyzer
  ClassAnalzyer.cpp
  MyFrontendActionFactory.cpp
  MyFrontendAction.cpp
  MyASTConsumer.cpp
  )
target_link_libraries(class-analyzer
  PRIVATE
  clangAST
  clangFrontend
  clangTooling
  )

The first couple lines tell the build system that the compiler should be /usr/local/bin/clang++ (the one just built in the previous steps).

The next add_clang_executable section tells the build system which source files to build as a part of our executable. We'll get more into the details of what each source file does soon. It also tells defines the name of the executable for the build system. Here it is called class-analyzer since it analyzes class names.

The target_link_libraries section informs the build system about the Clang front end libraries we should be linking against. These are the libraries which really expose the power of Clang’s AST to the tool we'll build.

Clang's API documentation is a good place to start looking for hints on how we should start writing the class-analyzer tool. Another good place to start is by scanning the source code of the Clang project we cloned earlier, for other tools! [clang-tools-extra](https://github.com/llvm/llvm-project/tree/main/clang-tools-extra) has multiple examples – these have been a source of inspiration for the code written here.

So now, let's start with the code for our very first source file. This file is contains the main() function of the executable. It looks something like this:

#include "clang/Tooling/CommonOptionsParser.h"
#include "clang/Tooling/Tooling.h"

#include "MyFrontendActionFactory.h"

#include <memory>

using namespace clang::tooling;
using namespace llvm;

static llvm::cl::OptionCategory toolCategory("class-analyzer <options>");

int main(int argc, const char** argv)
{
    // Use clang's argument parser infrastructure
    // This is used for giving clang tooling the path
    // to the source files passed in to the tool.
    // It also gets the compilation database - a collection
    // of the compiler options used in the invocation of the tool
    auto argsParser = CommonOptionsParser::create(
        argc, argv, toolCategory);
    if (!expectedArgsParser) {
        llvm::errs() << argsParser.takeError();
        return -1;
    }
    CommonOptionsParser& optionsParser
        = argsParser.get();
    ClangTool tool(optionsParser.getCompilations(),
                   optionsParser.getSourcePathList());
    auto myActionFactory
        = std::make_unique<MyFrontendActionFactory>();

    return tool.run(myActionFactory.get());
}

This source file essentially creates a tool which runs a clang FrontendActionFactory. Now to understand what FrontendActionFactory does, let's take a look at Clang's documentation for it.

We see that it has a pure virtual method,

virtual std::unique_ptr<FrontendAction> create () = 0;

which returns an std::unique_ptr to a [FrontendAction](https://clang.llvm.org/doxygen/classclang_1_1FrontendAction.html) object. FrontendAction is, in its essence, a class which allows callers to perform custom actions as Clang parses the AST of a translation unit given to it. A translation unit in simple words is the combined code given to the compiler to create an object file. It contains code included through all the header files + code in a C / C++ source file

This will become clearer as we get further along in the article.

Now we come to writing our own FrontendActionFactory which you can call MyFrontendActionFactory. This is a very simple class which just overrides the create() virtual method. It looks like this:

// Header file MyFrontendActionFactory.h
#pragma once

include<clang/Tooling/Tooling.h>


class MyFrontendActionFactory : public clang::tooling::FrontendActionFactory{
    public:
    MyFrontendActionFactory();
    std::unique_ptr<clang::FrontendAction> create() override;
};                                                         

// Source file MyFrontendActionFactory.cpp

#include "MyFrontendActionFactory.h"
#include "MyFrontendAction.h"

MyFrontendActionFactory::MyFrontendActionFactory() {

}

std::unique_ptr<clang::FrontendAction> MyFrontendActionFactory::create() {
    return std::make_unique<MyFrontendAction>();
}

Since MyFrontendActionFactory::create() needs to return an std::unique_ptr to clang::FrontendAction, we'll need to create a clang::FrontendAction object.

If we look at the Clang documentation for FrontendAction, we'll be particularly interested in looking at what we can do with the AST (Abstract Syntax Tree) of the source.

We might spot the following method:

virtual std::unique_ptr< ASTConsumer >
CreateASTConsumer (CompilerInstance &CI, StringRef InFile) = 0;

This is a virtual method that a class which inherits from FrontendAction can implement. It returns an ASTConsumer which according to the documentation,

"...is an abstract interface that should be implemented by clients that read ASTs."

So, this method looks really promising if we want to create something that will let us read the Clang generated AST!

If we look at the FrontendAction documentation again, it shows us that ASTFrontend is a class that inherits from FrontendAction. We also learn that it is:

"The Abstract base class to use for AST consumer-based frontend actions."

It only has one pure virtual method: CreateASTConsumer(). This seems promising, since...we might be able to create our own ASTConsumer object.

So, we start by reading through ASTConsumer's documentation. We see that it has a virtual method

virtual void
clang::ASTConsumer::HandleTranslationUnit(ASTContext &Ctx)

where the documentation states:

"HandleTranslationUnit - This method is called when the ASTs for entire translation unit have been parsed".

This is exactly what we want. We can override this method to do interesting things with the parsed AST.

You might now be wondering – how exactly can we use the parameter passed to this function ASTContext to actually go through the AST?

There's a class in the Clang front end API which can help us here: RecursiveASTVisitor. This is a class that does a depth-first traversal of the Clang AST and visits each node. It has methods such as VisitDecl(), VisitStmt() and so on which can help us go through virtually the whole source file's AST.

It also has a method which is particularly interesting: TraverseDecl(). This method recursively traverses through all the declarations starting from the root declaration given to it.

Putting it All Together in Code

So now what we need to do is give TraverseDecl() the root declaration of our translation unit and it will traverse the entirety of it. We can define special 'hooks' which will get called as this traversal happens. One such hook is:

bool VisitRecordDecl(const clang::RecordDecl *record);

This is called each time the RecursiveASTVisitor traverses through a CXXRecordDecl – which is Clang speak for a C++ class. We'll overload this method with our own version to do something interesting: getting the C++ Class' name and seeing if it starts with an upper-case character.

Putting all this together, here's what we get:

// MyFrontendAction.h header file
#pragma once

#include <clang/Frontend/FrontendAction.h>

class MyFrontendAction : public clang::ASTFrontendAction {
    protected:
        std::unique_ptr<clang::ASTConsumer> CreateASTConsumer(clang::CompilerInstance &ci, llvm::StringRef file) override;
};    

// MyFrontendAction.cpp source file
#include "MyFrontendAction.h"
#include "MyASTConsumer.h"

std::unique_ptr<clang::ASTConsumer> MyFrontendAction::CreateASTConsumer(clang::CompilerInstance &ci, llvm::StringRef file) {
    return std::make_unique<MyASTConsumer>(ci, file);
}

//MyASTConsumer.h header file

#pragma once

#include<clang/AST/ASTConsumer.h>
#include<clang/Frontend/CompilerInstance.h>

class MyASTConsumer : public clang::ASTConsumer {

public:
    MyASTConsumer(clang::CompilerInstance &ci, llvm::StringRef file) {}
    void HandleTranslationUnit(clang::ASTContext &context) override;
};

// MyASTConsumer.cpp source file

#include <clang/AST/RecursiveASTVisitor.h>
#include "MyASTConsumer.h"

#include <iostream>

static bool isFirstLetterUpperCase(const std::string &str) {
    return str.size() != 0 && std::isupper(str[0]);
}
class MyASTVisitor : public clang::RecursiveASTVisitor<MyASTVisitor> {
    public:
    bool VisitCXXRecordDecl(const clang::RecordDecl *record) {
        std::string name = record->getNameAsString();

        if (!isFirstLetterUpperCase(name)) {
            std::cout << "Record Decl : " << name
                      <<" doesn't start with uppercase! \n";
        }

        return true;
    }
    bool TraverseDecl(clang::Decl *decl)  {
        return
           clang::RecursiveASTVisitor<MyASTVisitor>::TraverseDecl(decl);
    }
};

void MyASTConsumer::HandleTranslationUnit(clang::ASTContext &ctx) {
    clang::TranslationUnitDecl *tuDecl = ctx.getTranslationUnitDecl();
    MyASTVisitor visitor;
    visitor.TraverseDecl(tuDecl);
}

Now to build, we just do:

cd ~/ast-analyzer/build/; ninja class-analyzer

This builds the class-analyzer executable in the build/bin directory.

Now to test out the analyzer, we create a test.cpp source file:

// test.cpp
class Test {
public:
 int a;
};

class testLower {
public:
 int b;
};

int main() {
        return 0;
}

Run class-analyzer on it:

bin/class-analyzer test.cpp

The output of this command is :

Record Decl : testLower doesn't start with uppercase!

We can use a multitude of such Visit* methods such as VisitEnumDecl, VisitFunctionDecl, VisitVarDecl, and so on to get valuable information about the source file and create our own tools. Just think about any tool which runs and performs actions on code or gives suggestions to the user.

You may think that this seems like a lot of work for a small task. But think about the potential. For example, you could write a tool which automatically gives a user suggestions to improve their code style. Or you could create a tool which analyses C++ code and finds lines of code where there might be security vulnerabilities.

The possibilities are endless. Clang's front-end libraries are extremely powerful and you can build many cool projects and tools with them.

Summary

In this article, you learned how to get and use the rich collection of Clang's Front-end libraries to parse a C++ source AST. You can use these libraries to write interesting static code analysis tools.

As this article showed, one of the most important parts of the journey of exploring Clang's libraries is the art of reading the API documentation and applying it to the problems your tools aim to solve. I hope you enjoyed the article!