Then there are more complex analyses that you'll need to do. At times relationships between two seemingly unrelated data sets can provide surprising insights and unveil important opportunities for the organization.

Data scientists and engineers are continuing to improve how they break down and work on data. Experimentation entails discovering the right correlations among data points.

This means they also need to do some sort of parallelization of such data and resulting models. Parallelization simply means that the same data set is being operated upon in many different ways without damaging the integrity of the original data.

In this article we are going to talk about how you can make sure you're doing such experimentation and parallel processing efficiently and that it provides the maximum insights. We will be tackling different concepts related to data storage and data versioning.

Block Storage vs Object Storage

For the uninitiated, we first must understand the difference between block and object storage and why the latter is the better option when dealing with data experimentation.

_Image source_

What is Block Storage?

It is called “block storage” (also known as SAN) because each dataset (in the form of files) is grouped into blocks stored in disks.

A classic example of block storage is the file system on your personal computer. For enterprise-level use-cases, it is scaled through a network of hard drives connected through fiber optic cables.

There are a few disadvantages to using block storage. First, if a sector (or a block) becomes corrupted, it can damage the files. Another problem is the lack of scalability (expanding the network of fiber optic cables is costly).

What is Object Storage?

In object storage, data is stored as objects. Each object contains the actual data, called the blob, a unique identifier (UUID), and metadata, which contains information about the object (such as timestamp, version, and author).

Object storage makes it cost-effective to scale your data store—you don’t need complex hardware for this. It also makes data retrieval faster as each object can be retrieved through its UUID.

This is in contrast to block storage, where each data location needs to be identified before the actual information can be retrieved.

One disadvantage of using object storage is that data can only be written once and cannot be updated. But this isn’t really a disadvantage as we will see further on in this article.

What Problems Does Object Storage Solve?

As we have already seen, data retrieval can be incredibly fast with object storage (no matter the size of the data store). But when it comes to data experimentation and data parallelization, object storage shines the brightest.

As mentioned before, you can't overwrite any data already stored as an object. This ensures object storage is protected from unwanted (or unauthorized) data destruction or updating. That’s great to know if you do a lot of data processing where accidental corruption of information could happen.

One other problem that object storage can solve is that it doesn’t require data to be structured. As companies produce and consume tremendous amounts of information every moment, often non-structured data (such as PDFs, videos, images) are not so easily processed into useful forms (such as for analytics or dashboards).

With object storage, this is now possible. You can now use non-structured data to develop machine learning models.

With data storage, it’s possible to have different versions of the same blob (with different metadata). As there is Git for code version control, we can have similar ways of managing different versions of the same data.

This brings us to the concept of data lakes.

What are Data Lakes?

Data lakes are central repositories of data that don’t care which format such data is in.

Companies produce and consume tremendous amounts of data. Such data traditionally sits in silos because they belong to different departments or are in different forms (for example, videos aren’t stored in the same directory as the data in the MySQL database).

With data lakes, any department in the enterprise can store information without the need to pre-process it. Likewise, any data can be retrieved and analyzed by anybody from any department.

Data lakes are important because they make data analytics extremely fast and convenient.

How Data Experimentation and Parallelization Work with Object Storage

As with developing software, working with data requires us to utilize tools that can aid us in our workflow. A powerful open source tool for experimenting with data and performing parallelization (that is working on the same data to create different sets of machine learning models) is LakeFS.

LakeFS is an open source platform that provides Git-like capabilities when working with data. This means you can create branches (allowing you to experiment with data) and commit versions of data (and data models).

Why is this Git-like feature important?

First, you need to make sure that your data lake is ACID compliant. This means that your data changes can happen in isolation (in branches). Thus, the integrity of the data is maintained in the master branch (until such changes are ready to be merged).

Another important feature of LakeFS is continuous integration of data (again, much like in software development). Enterprises need to incorporate new data quickly and without being disrupted. Therefore, this ability to have a CI/CD workflow is invaluable.

So, let’s see how we can get started with using LakeFS with our object storage experimentation and parallelization.

How to Install LakeFS

Locally you can install LakeFS by running the following command in your terminal:*

Code source

_*This is assuming you have Docker and Docker-Compose installed in your system. If you don’t have Docker and Docker-Compose, you may try other installation methods here._

Now visit http://127.0.0.1:8000/setup in your browser to verify you have installed it correctly.

How to Create a Repository in LakeFS

Once you’ve verified that LakeFS is installed correctly, go ahead and create an admin user.

Image source

_Image source_

Click on the login link and log in as an administrator.

On the page to which you get redirected, click on Create Repository. A popup will appear:

_Image source_

Congratulations! You now have your first repository. This is the main “bucket” in which you are going to store your data.

Next, we’ll start adding some data.

How to Add Data to your LakeFS Repository

Visit here to install AWS CLI.

With the credentials created during the admin-user creation phase, configure a new connection profile:

Code source

To test if the connection is working, run the following:

Code source

Now, to copy files into the main branch:

Code source

Just note that we need to prefix the path with the name of the branch we want to use.

Now, we will see the file we’ve added in the UI:

_Image source_

Next, we will need to know how to commit and create branches. To do that, we will need to install the LakeFS CLI.

How to Install the LakeFS CLI

You need to first download the binary file here.

Again, we need to use the earlier created admin credentials:

Code source

Here are some of the commands we can run to try it out:

Code source

You can find all the other commands, such as branch creation, and so on, online.

There you have it! Now, you can work with your data any way you like. Experiment without guilt and create multiple versions of your data models.

In Closing

In this article, we covered a bit of ground. We learned the different kinds of data storage mechanisms and why object storage has a lot of edge when dealing with data experimentations and parallelism.

Next, we looked into data lakes and LakeFS, which is a powerful tool for working with data.

At first, it might seem a daunting task. But, as we’ve shown here, with the right set of tools and knowledge, there’s a lot you can accomplish.

IT Managers earn an average of $91,000 a year. This salary bracket can actually range from $21,000 to $180,000 annually, depending on the industry of the IT Manager. In each case, though, the job entails problem-solving and project management.

Specifically, an IT Manager position has the following job description:

• Analyze, plan, and design the company’s information system needs and execute such plans and designs.
• Monitor and improve such a system (which may also include the company’s full telecommunications infrastructure) and recommend any upgrades required to its top management.
• Manage the company’s IT policies, including information policy, data/information access, and regulatory compliance.
• Hire and lead system administrators and other personnel who will be part of their team.
• Identify the training needs of the team and be able to arrange for such training and measure the benefits.

The first three above, as you will see, require the IT Manager to be astute in the technical aspects of the job. In particular, they must be competent in network-related areas. They should master these skills if they want to reach the higher tier of the salary scale.

Luckily, there are many training programs designed for IT Management professionals. These aren’t just regular training programs but fully-fledged certification courses. What follows are some of the top programs.

Cisco Certified Network Associate (CCNA)

The Cisco Certified Network Associate (CCNA) course is probably the most widely respected certification out there for IT professionals. It is an intermediate-level certification course. A CCNA holder has high competence in configuring, operating, and troubleshooting routed networks.

To get this certification, you must take and pass the CCNA 200-301 examination, which deals with fundamentals on network security, internet protocols, automation, and other security-related skills.

The exam is composed of 120 questions where each question has an associated score. To pass the exam, you must have a score of at least 800 (the highest attainable is 1,000). The cost for this exam is $300.

Cisco Certified Network Professional (CCNP)

Whereas CCNA covers the basic to intermediate skill level, the Cisco Certified Network Professional certification is for those who want to achieve a higher level of certification.

Unlike the CCNA, which requires the candidate to pass one exam, CCNP certification requires two exams: a core exam, covering fundamental networking concepts, and a concentration exam in any one area of intended focus (network design, automation, and so on).

It’s best to get a CCNP certificate after at least three years of experience in this industry. You don't need to take the CCNA before the CCNP examination. The costs for CCNP examinations range anywhere from $900 to $1,200.

Secure Access Service Edge (SASE)

SASE is a new technology (as described in Gartner’s “The Future of Network Security is in the Cloud”). This architecture integrates Wide Area Network (WAN) and some native security protocols of the cloud.

Cato Networks is a company that offers a cloud-based SASE platform, and they also offer a certification course for SASE so you can get familiarized with this new technology.

For this particular architecture, the center of a company’s network access is its physical data centers. For SASE, that center transfers over to the cloud.

In a nutshell, SASE makes it more efficient for IT Managers to administer the network as it removes some of the complexities associated with traditional physical data center setup.

To get a SASE certification, you need to take the SASE Expert Level 1 course and pass the associated examination (minimum score must be 85%).

CompTIA cloud+

CompTIA’s cloud+ certifications are for those who want to validate their system administration, cloud services maintenance, and other network-related knowledge and skills.

Currently, there are two such exams you can take to get a certification.

The CV0-002 (launched in February of 2018) is designed for candidates whose tasks include integrating various solutions into a business-specific system. It has 90 questions max and can be taken in 90 minutes. The passing score (from 100 to 900) is 750. The cost to take this exam is $329.

The CV0-003 (still in beta mode) is designed to get the candidate vendor-agnostic in creating solutions for their organizations. It has 110 questions that can be answered in 165 minutes. At the moment, there isn’t any finalized passing mark, and since it’s in beta, the cost to take this particular certification is just $50.

In both cases, it is recommended that the candidate has at least two years of experience.

AWS Certified Solutions Architect

The AWS Certified Solutions Architect certification is geared for professionals who design network architecture involving AWS. It requires that the candidate has at least two years of experience in operating and administering AWS applications.

It is one of the more sought-after certifications, as Amazon is the top vendor for cloud computing services. Hence, more and more organizations hire professionals that can navigate through this infrastructure.

With a price tag of $300 to take the exam, you should prepare for it thoroughly. The great thing is Amazon provides some preparatory materials for those interested in acquiring this prized certification. And so does freeCodeCamp.

Salesforce Certified Development Lifecycle and Deployment

As Salesforce is a key enterprise CRM solution, organizations often count on this SaaS platform to manage their relationship with customers as well as provide centralized access for related departments.

This exam also has a hefty price tag - $400. The exam itself has 60 multiple choice questions and requires the candidate to answer at least 60% of them correctly. Those failing the exam are allowed to retake it for a $200 fee.

In conclusion

These certifications are by no means the only ones that can enhance an IT Manager’s skills (and hence, career prospects). There are also others related to project management.

What is important to note here is that in the ever-changing technology landscape, it’s always essential to put yourself into continuous learning mode.

Yahoo, however, has a lot of limitations as it only provides location-specific current data and ten-day historical data.

If your application requires more features and information, using this free API may not be the best option.

What follows is a list of the best alternatives to the Yahoo weather API along with their key benefits and features, most common use cases, and pricing.

ClimaCell Microweather API

ClimaCell’s Microweather API provides a wide range of information offered both as numerical data and map layers. It can provide up to four weeks of historical data from partner stations and up to six hours of historical data from its own observations.

Key features/benefits of the ClimaCell Microweather API:

• Artificial intelligence-driven data model that processes data from a wide range of sources such as drones and IoT devices
• Map layers for more than 20 parameters
• Hyper-local and global historical and real-time data
• Out of the box templates for easier crafting of request calls

Best used for:

• Most corporate applications. The API is used by companies such as Ford and Uber.

Pricing: Free plan allows for up to 1,000 calls per day. Paid plans start at $87 per month.

OpenWeatherMap API

OpenWeatherMap is an API inspired by Wikipedia’s crowdsourcing model. It is probably the most popular weather API because of its simplicity.

Key features/benefits of the OpenWeatherMap API:

• Very simple to use
• 40-year historical weather data for any location
• Useful data visualization such as a “this-day-in-history” chart
• 16-day forecast for several locations
• Three-hour and five-day forecasts
• Very accurate data as it collects from over 40,000 stations globally
• Creative Commons Share-Alike licenses on weather maps which allow developers to easily build map-based applications

Best used for:

• Personal weather stations and simple projects for developers
• Applications geared toward the agricultural sector

Pricing: Its free tier allows for up to 100 calls per day. Paid plans start at $40 per month.

Accuweather API

Accuweather is another very popular weather API. It processes over 30 billion requests daily and provides imagery endpoints.

Key features/benefits of the Accuweather API:

• Developers can quickly get started with its Apigee portal (Accuweather’s project with Google).
• Its “Indices API” provides other useful information such as flight delays, mosquito activity, data for stargazers, and a host of daily indices for specific locations.
• It provides engaging video presentations along with the minute by minute forecasts.
• As its name suggests, it provides super accurate weather data for almost any point on earth.
• Weather information can be customized.

Best used for:

• It’s the best option if the application needs to have video presentations.
• Applications geared toward managing disaster risks.

Pricing: It has a limited trial that allows for 50 calls daily. Paid plans start at $25 per month.

Stormglass API

The Stormglass weather API provides high-resolution forecasts globally from renowned meteorological institutions. It allows developers to fetch data from any coordinate through simple HTTP requests.

Key features/benefits of the Stormglass API:

• It provides up to 10-day forecasts with high-resolution.
• Provides marine information such as tides, water chemical analysis.
• Provides data for renewable energy (wind and solar indices) as well as agricultural information (soil moisture, surface temperature).

Best Used For:

• Marine-related applications.
• Renewable energy monitoring.
• Agricultural organizations.

Pricing: It has a free tier that allows up to 50 daily API calls. Paid plans start at 16 euros a month.

Weather Company API (IBM)

The Weather Company API is owned by IBM and is part of a much more comprehensive data package. Its main value proposition is the provision of highly actionable insights that allows business to optimize their operations and reduce costs while increasing the bottomline.

Key features/benefits of the Weather Company API:

• It provides very accurate forecasts of about 500-meter resolution.
• It has an extensive number of information sources including high-resolution observations from 250,000 weather stations.
• It provides historical data encompassing a 35 kilometer worldwide grid allowing businesses to understand patterns to make better decisions relating to weather.
• It provides real-time severe weather forecasting.
• It also provides non-weather information such as air-quality, pollen and flu indices.

Best Used For:

• Business relying heavily on quick forecasts as well as long historical weather data.

Pricing: Free trial that allows for up to 10 calls per minute. The trial then expires once the limit of 10,000 free calls is consumed. Paid plans start at $24.95 per month.

Weather 2020 API

Weather 2020 prides itself on being a predictive weather modeling platform. Its main proposition is that it can provide for up to nine months forecast just based on the preceding three-month observation. Its statistical model is based on the work of famed meteorologist Gary Lezak.

Key features/benefits of the Weather 2020 API:

• It provides long-term forecasts allowing businesses and organizations to have longer planning horizons.
• It offers over 10-million forecasts per day and is best used for mobile applications.
• It has a very interactive dashboard that displays key information, analytics and graphs.

Best Used For:

• Mobile weather applications.
• Creating video forecasts (available for paid plans).

Pricing: The free plan allows for up to 1,000 API calls per day. Paid plans start at $ 9.99 per month.

Conclusion

As we’ve seen from the list above, each option has certain use-cases for which it is best suited. This is, by no means, an indication that these are the only uses these APIs have.

What you select will depend on several factors including cost, amount and accuracy of data required, as well your own preferences.

As you can imagine, there are many different storage options.

In this article, we’ll briefly discuss two of the most used cloud solutions: block storage (also known as SAN or storage area network) and object storage. After this, we will go through my top 5 suggested object storage solutions.

There is a third type of storage that’s commonly used: file system storage. However, this mechanism can also concur with SANs and object storage, so we won't go too deep into it.

What is Block Storage?

Block storage is a network of hard drives connected via fiber-optic network. This gives it an edge over copper cables due to the increased speed.

The reason it's called block storage is that each file in this system is divided into “blocks” of data stored in a disk. Sectors in the disk hold onto individual blocks of data, and these blocks, when combined, form the whole file.

So while there are advantages of using SAN, like high scalability, it is costly and can get incredibly complex as the network grows.

What is Object Storage?

The defining feature of object storage is that, instead of storing files as blocks, data is stored as objects.

Typically these objects will have more data attached to them than the blocks used for block storage. The objects often include:

• A blob which contains all the payload (i.e., image, video, text content)
• Metadata, which tells us more about the file (timestamps, permissions, author, revision, and so on)
• A universally unique ID (UUID)

One major advantage of this type of storage is that objects are easily obtained and found because of their UUID. With block storage, there’s a specific hierarchy of files that a user goes through before getting the data they need, which can considerably slow down data retrieval.

Now that we have that out of the way, here’s a list of my top 5 object storage tools for developers:

Amazon AWS S3

S3 is one of the pioneers of object storage. It manages gigantic loads of data from all over the world across hundreds of industries.

Features:

• High reliability and durability as it stores S3 objects in copies across multiple systems.
• Allows you to manage costs through its S3 Storage Classes, which provides different rates depending on access patterns.
• Provides the highest security and protection for your data.

Google Cloud Storage (GCS)

Google offers four different storage types for business levels of all sizes. When you’re moving data across each of those storage types, it will provide you with the data lifecycle. With this, you can manage how long data should be stored until it has to be deleted.

Features:

• You don’t have a minimum object size.
• You have access to storage locations all around the world.
• Very high durability and low latency.
• Data has redundancy across several geographic locations.

LakeFS

LakeFS is an open-source tool that works with object storage data lakes. Data lakes usually store files or blobs in raw format centrally through a repository.

Data lakes, on their own, are limited by the lack of frequent communication between entities. LakeFS solves this by using data versioning.

Features:

• Through S3 or GCS, it allows scaling up to Petabytes in size by using a system that mimics Git.
• You can experiment as it provides you with a development environment with your data.
• Since it uses a Git-like scheme, you can safely use new data in another branch without affecting the main branch. You can then, later on, merge it safely once each aspect of new data checks out (schema, etc.).

MiniIO

MiniIO is another open-source solution. It utilizes the Amazon S3 API, which makes it perfect for high scale projects that require super strict security.

Features:

• It calls itself the world’s fastest object storage as it has a read/write speed of up to 183 GB.
• It applies web scaling principles – a cluster can join forces with other clusters until it forms multiple data centers.
• It’s Kubernetes friendly.
• Because it's open source, users can improve and freely redistribute it.

Stackpath

StackPath offers both a Content-Delivery Network service, Edge Computing, and an S3 compatible Object Storage. It touts itself as a cheaper option to Amazon S3 and other cloud providers.

Features:

• It is six times faster than competing services, especially when combined with the CDN or the Edge Computing platform.
• It is serverless, which means it needs no warmup.
• It has 45 edge locations, which means your application is available worldwide with the same performance anywhere.

In Closing

There you have it – a short list of the top object storage tools that you can use for your next web or mobile project. Object storage has indeed proven a great way to store data when scalability is the most significant consideration.

Thanks for reading this article! I hope you learned a thing or two about storage models, especially about object storage. Please feel free to connect with me on LinkedIn and Twitter.

The success of using TDD, however, depends highly on how productive the developer can be while implementing code and application testing.

This is where automation tools come in.

Below is a list of 10 recommended testing automation frameworks/platforms and a short summary of their features.

Selenium

Selenium is, arguably, the most popular automated testing framework. It is a browser-based framework that works on different operating systems and browsers.

It features the Selenium IDE, which is Chrome and Firefox extension that allows the recording and playback of user interactions. For scaling tests (i.e, running on several machines) then Selenium Grid is the tool of choice.

The most popular tool in the framework is Selenium Webdriver, which is a collection of language specific bindings and allows for the creation of browser-based regression tests.

Perfecto.io

Perfecto is both a web and mobile app testing framework. It’s delivered as a SaaS tool and allows testing engineers to run their tests from anywhere.

The web testing tool provides for parallel testing of apps on different operating systems and browsers. It allows for a huge volume of daily tests to be run, and the tests are 50% faster than any other framework.

The mobile test tool allows engineers to test on both emulators and actual devices. Just like the web test tool, it allows for a high volume of test runs daily (10,000 executions).

Serenity

Serenity BDD’s slogan is “Automated Acceptance Testing with Style”. This is because Serenity’s unique angle is helping testers write world-class test reports and documentation.

Testing starts with the creation of user stories and acceptance criteria. Serenity then automates the acceptance criteria. Tests are automatically broken down into steps that make them more readable. Developers can easily run these tests against actual application executions.

Once tests are finished, Serenity creates detailed reports that include screenshots of the tests and all relevant information such as error messages and execution times.

Cypress

Cypress is a great tool to use for frontend or end-to-end automated testing.

Installing Cypress in a project is as easy as running a simple npm install cypress or yarn add cypress command. JavaScript developers who are used to unit testing tools such as Jest or Mocha will find writing Cypress test scripts a breeze.

Its dashboard makes testing more powerful and faster by allowing tests to be grouped by browser type, environment, package type, and so on. The parallelization feature allows developers to run more tests and test more features easily.

Lambda Test

LambdaTest is a leading test automation software for both desktop and web apps.

It allows for live and interactive testing of both publicly hosted and local machine hosted web applications and websites.

LambdaTest also allows developers to run Selenium test scripts with its Browser Testing Grid. It is a very powerful and versatile suite that also integrates with CI/CD tools such as Jenkins, Circle CI, and Travis CI.

Now the coolest part is that it allows for geo-testing, meaning web applications can be tested for how they perform depending on which location in the world they are being accessed from.

TestProject.io

TestProject’s community is probably one of the biggest among those in the testing community. It is an end-to-end and API cloud hosted test framework.

As an open-source project, users get easy access to updates and can even participate in its improvements. It is ready for use with Selenium with all of its pre-packaged dependencies.

With its myriad of integration tools, it allows for testing on different browsers, and even in Docker.

One of the best things about TestProject is it allows non-coders to perform testing through its Scriptless Test recorder.

Katalon Studio

Another open-source API, web, and mobile testing suite is Katalon Studio. It is one of the most complete testing suites in the entire list, with features like recording, auto-generating test scripts, and powerful integrations.

It is a testing framework that is easy to start (yes, even for non-coders) but is powerful enough for scaling.

Katalon Studio works well with existing CI/CD setups. For instance, it's easy to integrate it with Github or Gitlab’s continuous testing tools. This makes it great for Agile teams.

OpenTest

OpenTest is another open source tool that automates testing for API’s, web, mobile and desktop applications. It is mostly used for functional testing.

It still has lots of limitations as there are some test scenarios that are not yet supported. However, it is very good for beginners and non-coders alike since it uses plain English keywords for writing test actions.

Some of its most notable features include web testing with Selenium, mobile testing with Appium, keyword-driven testing, parallel testing, and data-driven testing.

AccelQ

AccelQ is a codeless test automation platform that is artificial-intelligence based. As such, it is best used for Agile development as it allows for rapid building and easy introduction of changes in the project.

The platform allows for API and end-to-end testing. The AI backbone provides predictive analytics that help in accelerating the test script generation. It has an intuitive and easy to use UI that allows even beginners to get started quickly.

Conclusion

The choice boils down to, not surprisingly, your particular use-case.

However, given the extensive capabilities of each of these tools, it is understandably difficult to select one. The beautiful thing about a lot of them being open source is that development teams can use one for a particular project, and another for a different one.

Here are a few tips you can use to help your team improve, become more efficient, and deliver on projects faster.

The Engineer’s Way of Improving Team Productivity

The importance of providing your team with the right tools cannot be overstated. It's not the only important factor, as we’ll see later, but it can definitely make certain processes go more smoothly.

For instance, since code debugging is one of the most tedious tasks a developer needs to perform, you can equip your team with tools that will help them debug faster and more conveniently.

Let’s say you observe that your developers, who are working remotely in different parts of the world, are getting stuck with debugging code.

Debugging your own code is challenging enough. But if you have to debug someone else's – and they live in a different time zone – it can cause massive delays.

Using a debugger can help manage issues like these and make the entire team faster and more efficient.

Go beyond the system – to the people

However, increasing productivity isn’t just about finding the right tools to get the job done. A dev team is a group of human beings, not just code. And humans are complex organisms.

Additionally, software developers are highly intelligent individuals. Thus, they tend to not perform as well if what they’re doing isn’t challenging.

The point is not to look at productivity just at the system level. You need to also see the people who are inside that system.

This is a big problem I see with most startup founders or dev team managers. Most neglect developing leadership skills.

A team of developers isn’t so different from any other kind of a team. It’s a group of individuals put together to accomplish a goal. No matter how “horizontal” you think your team should be, at the end of the day it will fall apart without leadership.

Having been an avid learner in the subject of leadership, I will be borrowing some wisdom from John C. Maxwell’s The 17 Indisputable Laws Of Teamwork.

Make Sure Your Team Has a Vision

A vision gives direction to the team. It provides a battlecry that will make each member emotionally invested in the team’s goals.

A vision also ensures that individual achievements aren’t meaningless trophies. Each member can now align their individual goals to what the whole team ultimately wants to achieve.

For instance, take a look at Apple's vision statement:

"We are on the face of the earth to make great products, and that’s not changing. We don't settle for anything less than excellence."

With this kind of vision, each team member will believe that each line of code they write contributes to creating great products and not settling for anything less than excellence.

Productivity naturally occurs, as each member will have a higher view of their contributions to the team.

Focus on Your Team’s Strengths

Having talented developers in the team doesn’t necessarily translate to excellence in performance. Each member of the team has a particular expertise that needs to be utilized in the right tasks for that expertise.

A team’s dynamic changes depending on how members are placed. For instance, place one person in the wrong task and you'll get a confused team. Or worse, it will generate frustration (and resentment).

But pairing the right people with the right tasks will create progress and a high level of motivation (and thus productivity) in the team.

So how do you ensure that you’re focusing on your team’s strengths? Here are a few tips:

1. You must personally be familiar with every aspect of the team’s vision and shared values.
2. Not only should you be intimate with the vision and core values of the team, but you have to know how to adapt the team depending on each situation. Rigidity and an unwillingness to adapt kills productivity.
3. It’s important that you, as a leader, know each member personally so you’ll easily understand which tasks they’re more suited for.

Constantly Work on Better Communication

This may sound cliché, but I cannot overemphasize the importance of communication.

Software developers, especially when they’re “in the zone”, wearing headphones, shut themselves off from the outside world. They do this so they can focus.

It’s great. Coding is an art. The coder is the artist. Any distraction ruins the masterpiece.

But this doesn’t mean we cannot introduce systems and tools that encourage everybody to speak their minds and share ideas.

Stand-up meetings provide for a short but very effective way for everybody to get updated on the team’s progress.

Communication tools such as Slack or Telegram ensure that information is shared in real time.

Github and Gitlab also have tools that allow teams to discuss issues and be able to resolve them.

For example, team members can review each other's code and ensure proper collaboration on improving shipped code:

Image courtesy of: https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/commenting-on-a-pull-request

Conclusion

The bottom line is that increasing your dev team’s productivity is a mixture of systemization of tasks (such as equipping them with the right tools) and enhancing their potential as human beings.

This not only will ensure you have a highly productive team, but also a group of happy individuals avoiding burnout and enjoying doing what they do best.

Using these open source libraries provides tremendous productivity benefits. However, it also comes with disadvantages – namely in relation to security.

Cyber-criminals and hackers have been increasingly exploiting vulnerabilities in applications and IT systems. It has, therefore, become more and more important to ensure that code bases minimize or totally eliminate vulnerabilities.

However, keeping tabs on all the vulnerabilities, let alone updated ones, in projects can be very daunting. That’s why in this article we’ll take a look at eight tools that automate the detection and fixing of vulnerable spots in a project.

DeepScan

DeepScan is a tool that analyzes JavaScript and TypeScript code. At its core, it not only inspects for code quality à la ESLint, but it also employs data-flow analysis and looks into the execution flow. Errors and quality issues are detected even without running the code.

DeepScan works with most JavaScript libraries such as React and Vue.js.

Teams can simply integrate their projects’ GitHub repository with DeepScan. Every time a push is made into a repository, DeepScan provides a real time report on test results.

One of the best things about this is that code quality standards are more enforceable. DeepScan motivates teams to write quality code by grading the project as Poor, Normal, or Good.

SonarQube

SonarQube is an open-source platform that continuously inspects a project for code quality, bugs, code-smells and even security vulnerabilities.

It's a tool written in Java but has the ability to analyze other languages through the use of plugins.

Unlike most of the others in this list, SonarQube isn’t integrated into a project as a simple GitHub extension. It needs to be installed in the local machine for you to be able to use it.

It works by receiving the project’s files as input and then making the necessary analysis. It then generates data based on the analysis, stores that data in a database, and displays it in a dashboard.

Dependabot

Dependabot is a tool you use inside of GitHub that automatically creates pull requests upon detection of vulnerabilities.

This tool performs scans on all of a repository’s dependency files and searches for outdated or insecure dependencies. It then generates a single pull request for each outdated or insecure dependency. The developer can then check those pull requests and merge as necessary.

The great thing about Dependabot is that it’s owned by GitHub so that it can be seamlessly integrated into any repository. It performs constant monitoring and quickly updates users when there's a new vulnerability.

It can be very chaotic to receive daily notifications, so users can configure the frequency that the tool performs scans and creates pull requests.

SourceClear

SourceClear is a tool that helps developers understand more about the open source libraries they are using. SourceClear provides information on those libraries such as who created them, what they do, and which dependencies of such libraries have vulnerabilities.

SourceClear meshes well with a developer’s workflow and provides real-time reports on open-source code risks. It has machine-learning tools that makes it possible to provide such detailed information for each library used.

One of its main features is the prioritization of vulnerabilities that are directly in the code’s execution path. This can reduce remediation time for big projects by up to 90%.

SpotBugs

SpotBugs, a successor to FindBugs, is a static code analyzer for Java code bases. It can either be used as a stand-alone tool or integrated to other platforms/tools.

Most Java programs compile cleanly but are still buggy. Compilation captures mostly only syntax and references errors, among others. Usage of static analysis tools such as SpotBugs provides a more comprehensive solution in catching bugs and even vulnerabilities.

SpotBugs inspects Java bytecode (not the source code) and checks for bug patterns. It then classifies errors or potential errors based on how severe they are: Of Concern, Troubling, Scary, Scariest.

This tool is very good at identifying bug patterns (over 400).

Arxan Application Protection

Arxan Application Protection is a total solution to “protect apps inside and out”. This tool’s main selling point - Protecting applications against reverse engineering.

A lot of today’s attacks such as clickjacking are engineered by cyber-criminals through hacking the app’s binary code and then creating a replica app. Users are then lured into trusting this fake app and giving away their data such as banking passwords.

Arxan protects an application from such attacks by “hardening” the application’s code by inserting “code guards” into them. These code guards are tiny security units which protect the application and each other against compromise and they detect attacks at runtime.

GitLab

One of GitLab’s core value propositions to developers is that it is one of the most exquisite devops tools out there. Added to this is GitLab’s focus on secure deployment.

The platform has incorporated security in its already loaded devops arsenal. Developers can focus on coding while being confident that any security vulnerability will quickly be detected. This makes it very pleasant to use, as no additional tool or integration is necessary.

It employs what it calls the Secure Stage where all the security parts of devops are performed. This “stage” has a goal of identifying proactively any vulnerabilities before they can be exploited in production code.

Conclusion

Each tool has its own pros and cons and the choice to use one over the other depends on the particular taste of the developer. Often, some tools can even be used together.

The bottom line is that nowadays we are more equipped to handle security issues before they become big problems in our application projects.

In my opinion, one of the most powerful programming techniques is recursion. For readers new to programming — it is simply a function or method calling itself. To make the introduction simpler we have a method below that calls another method:

def outer_method(name)       (R1)
  inner_method + name
end
def inner_method             (R2)
  "Hello "
end
print outer_method("Steve") -> #"Hello Steve"

In the above method outer_method, which takes in a string as argument, calls inner_method, which simply returns the string “Hello “ inside it. Recursion is similar in that, say in this case, outer_method simply calls itself:

def outer_method(name)              (R3)
  outer_method("hello ") + name
end (R3)

One caveat, though, with code R3 above — it will run until the computer complains that resources are not enough to keep processing the method. It’s like running an infinite loop except that infinite loops don’t necessarily raise exceptions. The reason for this is that code R3 doesn’t have a ‘terminal state’ or a point where it doesn’t ‘recurse’ anymore.

We can solve this by including a terminal state:

def outer_method(name)                 (R4)
  return name if name == "hello "
  outer_method("hello ") + name
end

The first line inside the method definition simply states that if the argument name is equal to ‘hello’ then simply return name. That will then ignore any line after it. Therefore in the second line, the code outer_method(“hello “) will simply give the string “hello “ to be added to whatever name is in the main argument. So the same print outer_method(“Steve”) will result in the output “hello Steve” as well.

OK then, that may not be the best example for describing recursion (as the recursive version in this case doesn’t have that much advantage over the non-recursive one). But working on the binary tree height problem, we will see that recursion is so much simpler to understand and faster to run.

For this discussion let me put again the same example as I showed in the previous article:

Figure 1: Simple binary tree

which we can represent as the following array:

tree = [1, 7, 5, 2, 6, 0, 9, 3, 7, 5, 11, 0, 0, 4, 0] (T0)

The indices of the left and right children of any sub tree can be determined as follows:

left child of tree[i] is at index 2*i + 1 (T1)
right child of tree[i] is at index 2*i + 2 (T2)

If you’re puzzled about how the figure above became the array following it, I’ll direct you to read the previous article on the iterative method for clarification.

And again the formula for calculating the height of a binary tree, as well as the heights of any of its sub trees, is:

height = 1 + max of(left_child_height, right_child_height) (T3)

Now with these we can outline the steps to develop a recursive program.

Step 0: Set default values — To make the initial method call simple, I always like setting default values for the arguments that will change during each recursive call. Since we will repetitively compute heights, our indices will always change.

For instance, to find the height of the root’s (tree[0]) left child we will need to call the method on that left child (whose index is at 2*(0) + 1). Therefore, our method definition will be:

def tree_height_recursive(tree_array,i=0) (S0.1)

to indicate that for the initial call we are calling it on the root element. This will simply allow us to call tree_height_recursive by inputting only the tree_array. However, this also means, as we will see in the simulation afterwards, we can find the height of any sub tree by simply including its index as the second argument in the method call.

Step 1: Find terminal state — At which point do we simply return a value and not do any further recursive calls? In our binary tree problem, the terminal state is at:

return 0 if tree[i].nil or tree[i] == 0 (S1.1)

It simply says that if the element at index i does not exist or if its value is 0 then simply return 0. Logically, a non-existing sub tree will not have any height.

Step 2: Find the height of the left child — this is where the magic of recursion starts to benefit us. We don’t need any fancy code. No more declaring another array to hold the height of each element. No more multiple variable definitions for height indices and the heights themselves, just:

right_child_height = tree_height_recursive(tree_array, 2*i + 2)

We simply pass the index of the left child as second argument. Can you see why?

We do the same for finding the right child’s height next.

Step 3: Find the height of right child — Likewise, we simply do a recursive call to our method but passing the index of the right child as second argument:

right_child_height = tree_height_recursive(tree_array, 2*i + 2)

Now that we have the heights of the left and right children, we can now compute the total height.

Step 4: Calculate and return total height — As code T3 states, we just add 1 and the height of whichever is taller between the left and right children.

total_height = 1 + [left_child_height, right_child_height].max (S4.1)

Since S.4 will be the last statement in our method, then the evaluated total_height will be returned. Remember that if the conditions in S1.1 hold true (our terminal state) then none of Steps 2–4 will run and the method will simply return 0.

The full method below:

Comparing this to the iterative method, the recursive version took 3 fewer steps and 4 fewer variable definitions. The code also (excluding empty spaces and comments) is 7 lines fewer. On top of it, the recursive code will run 2x faster (using the benchmark built-in Ruby module). This is a big advantage if we’re running the method on binary trees hundreds of levels tall.

Now let’s do the same simulation as we did before. For the tree at T0 we run the recursive method:

tree = [1, 7, 5, 2, 6, 0, 9, 3, 7, 5, 11, 0, 0, 4, 0]

puts tree_height_recursive(tree_array)-> #should give us 4

Note that since we have a default i=0 in our method definition we don’t need to specify the index here because we are finding the height of the whole tree. To make this simulation more intuitive we shall create an imaginary array called call_stack where push every call to tree_height_recursive.

So then when we call the method the first time (the main call), we store it in a temporary variable ht_0 and push it to call_stack:

ht_0 = height of tree[0] = tree_height_recursive(tree_array,i=0)

call_stack = [ht_0]

We then run Step 1:

tree[0].nil? -> #falsetree[0] == 0 -> #false, it is 2

Since this results in false, we go ahead to Step 2:

since i= 0, then 2*i + 1 = 2*0 + 1 = 1:

left_child_height = tree_height_recursive(tree_array,1)

Since we cannot readily determine this height so then we push it again to call_stack:

ht_1 = left_child_height = tree_height_recursive(tree_array,1)

call_stack = [ht_0,ht_1]

Then upon doing Step 3:

ht_2 = right_child_height = left_child_height = tree_height_recursive(tree_array,)

call_stack = [ht0,ht1,ht2]

We cannot proceed to Step 4 until all the items in call_stack have been evaluated by our program and popped off from call_stack (which should happen for every time each height has been evaluated).

So we will also do the same for each of the succeeding heights. For instance, to compute ht1 we know that we have to compute for its own left and right children’s heights too. So that means the method will be called for them too. So as not to prolong this article, the reader is invited to try this on paper.

Ultimately, the method will be called recursively with i = 14 as second argument. Thus, at this point, call_stack will be:

call_stack = [ht0,ht1,ht2,ht3,ht4,ht5,ht6,ht7,ht8,ht9,ht10,ht11,ht12,ht13,ht14]

Now we will evaluate each. Note that from tree[7] up to tree[14] the elements don’t have any children. So we can simply evaluate their heights as 1 or 0 (depending on whether tree[i] is 0 or not (where i ≥ 7):

ht14 = 0

ht13 = 1

ht12 = 0

ht11 = 0

ht10 = 1

ht9 = 1

ht8 = 1

ht7 = 1

Again, when these heights are evaluated we simply pop them off successively from call_stack. After which, call_stack will appear as follows:

call_stack = [ht0, ht1, ht2, ht3, ht4, ht5, ht6]

Now, to evaluate ht6 we must remember that it is the call to tree_height_recursive(tree_array, 6). Inside this call we also call on the method to compute for the heights of the left and right children of tree[6]. These we previously already evaluated as ht13 and ht14. So then:

ht6 = 1 + [ht13, ht14].max = 1 + [1,0] = 1 + 1 = 2

So we now evaluate ht5, which is the height of tree[5]. We know the heights of its children are ht11 and ht12

ht5 = 1 + [ht11,ht12].max = 1 + [0,0].max = 1 + 0 = 1

Doing the same for ht4 to h1 (again the reader is invited to do the confirmation on paper):

ht4 = 1 + [ht9,ht10].max = 1 + [1,1].max = 1 + 1 = 2

ht3 = 1 + [ht7, ht8].max = 1 + [1, 1].max = 1 + 1 = 2

ht2 = 1 + [ht5, ht6].max = 1 + [1,2].max = 1 + 2 = 3

ht1 = 1 + [ht3, ht4].max = 1 + [2,2].max = 1 + 3 = 3

Again, we pop out each height from call_stack as we evaluate it so after evaluating ht1 the call_stack appears as follows:

call_stack = [ht0]

Now evaluating ht0 is returning to the main call to tree_height_recursive, so this is the remaining Step 4:

ht0 = 1 + [ht1, ht2].max = 1 + [3, 3].max = 1 + 3 = 4ortotal_height = 1 + [left_child_height, right_child_height].max

Which will return 4 as the result of the main method call.

As I keep mentioning, doing this on paper whether during the algorithm formulation or during simulation will help a lot in understanding it. This same method can also be used to determine the height of any of the sub trees inside the tree_array, for instance to determine only the height of the tree’s left child:

puts tree_height_recursive(tree_array, 1) -> #will print out 3

Or any of the lower sub trees:

puts tree_height_recursive(tree_array, 3) -> #will print out 2

Wrapping up

The key takeaway in creating a recursive algorithm, in my perspective, is setting the terminal state. Again, this is the scenario wherein the main method will not have to do any recursive call to itself. Without this, the method will just keep calling itself until the computer blows up (hyperbolically speaking…). When we have the terminal state we can easily set the arguments for the recursive calls and know that our method will safely return the value we expect.

Finally, working on algorithms challenge our minds to think. As software engineers, or even engineers in general, our main task is to solve problems. We, therefore, need to develop our critical thinking skills.

If for a problem, our first option is always ‘google it’ and copy/paste other people’s code without fully understanding the problem and the copied solution, then we are defeating ourselves.

So my suggestion is always have pen and paper ready and not immediately type code when faced with an algorithm challenge. Simulate the problem for simple inputs then come up with the code after you determine the steps (like I outlined them above).

Follow me on Twitter | Github

One such data structure is a binary tree:

_Photo by [Unsplash](https://unsplash.com/photos/EwKXn5CapA4?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText" rel="noopener" target="_blank" title="">Jeremy Bishop on <a href="https://unsplash.com/search/photos/tree?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText" rel="noopener" target="blank" title=")

Oh no not that kind of tree, I mean this one:

.Figure 1: Simple binary tree

In simple terms, a tree is network of ‘nodes’. A node is an object whose properties include the data itself and pointers to its ‘children’. For a binary tree, the maximum number of children each node can have is 2. A binary tree will have a root node and at most two children. Each child is just a pointer to another tree object or it can be nil. Using a hash, this can be visualized as:

tree = {
 :data        => 1,
 :left_child  => [another_tree] || nil,
 :right_child => [another_tree_again] || nil
}

Before we go into height computations let us first find some uses for binary trees.

If you observe the directories or file structure in your computer, it follows a (albeit the more general) tree structure. Each folder can contain files (the data) and a number of other directories (which are not necessarily data in themselves but rather just addresses of such data contained within those sub directories). There are other use cases for binary trees that discussed better by other articles:

In Quora

Stack Overflow

Binary trees are a vast subject and there are so many things that I can write about them (such as the different ways to search through them — a future article perhaps?). However, here I will be very specific — computing the height of a binary tree.

The first thing to understand in relation to this, is that we can represent a binary tree using an array. But even though that’s possible, there are a number of ways to lay down each node and associate them (as an element in an array) to their respective left and right children.

For simplicity we’ll use the “breadth-first” method of flattening the tree. In ‘breadth-first’ we place the data contained in each node starting from the root. Then we go to the next lower level, laying down each node’s data from left to right. We go through all the levels until the lowest one.

If a sub tree has no left or right child then such child can be represented as 0, as long as the sub tree isn’t in the lowest level in the binary tree.

Figure 2: Modified binary tree from Figure 1.

tree = [1, 7, 5, 2, 6, 0, 9, 3, 7, 5, 11, 0, 0, 4, 0] (T0)* array representation of Figure2

Numerically, we can compute the positions of the left and right children of each node:

left child of tree[i] is at index 2*i + 1 (T1)right child of tree[i] is at index 2*i + 2 (T2)

As we can see from figure 2 we can tell how tall a tree is — that is we just need to count how many nodes there are from the root down to the lowest element (including the root and the lowest element) along the longest branch. But when it’s already in array form, how do we know how tall it is?

First we must have a general formula for the height of any tree:

height = 1 + max of(left_child_height, right_child_height) (T3)

For multilevel trees then we can conclude that in order to compute the height of any sub-tree (and the tree itself) we first must compute the heights of the left and right children and then find the higher between the two. In computing the heights of these two children we need to compute the heights of their respective children and so on.

Having this we can now begin to outline an algorithm for computing the height of multilevel binary trees. There are two methods we can take, one is using iterations or loops, and the other, because of the repetitive nature of the steps (previous paragraph), is using recursion. I will follow up this article with a discussion on how to use recursion to do this. However, that would be too easy. So let’s learn the hard way first: we’ll do this using iteration.

Iterative Method

We will use the tree array T0 above to illustrate this process

Step 0: Declare a heights array which will store the heights of each sub tree.

heights = [] (S0.1)

Step 1: Iterate through the array — since we need to compute the heights of the descendants first, we iterate from the last element. And instead of using each method directly in the tree array we will use it for the indices of each element.

(tree.length - 1).downto(0) do |i| (S1.1)

Step 2: For each element, find initial height — if the element is zero (meaning it’s actually a nil node) then initial height is 0, else it is 1.

initial_height = tree[i] == 0 ? 0 : 1 (S2.1)

Step 3: Find height of left child — inside heights array, if the element has a left child then the height of this child is equal to:

left_child_height = heights[left_child_index] (S3.1)

In the above, the left_child_index can be computed as follows:

left_child_index = heights.length - i - 1 (S3.2)

I came up with S3.2 through a little trial and error. In the simulation that will follow these series of steps I will make mention of it.

To summarize though, I initially intended to unshift each descendant’s heights into heights so that the heights of each element would have the same indices as the element itself has on trees. But as I’ll later note, using unshift for this will be taxing resource wise for large array inputs.

So then I decided to use push. Each height will then be ordered in reverse compared to their corresponding elements’ order in tree. So that the height, let’s say of tree[0] will ultimately be located in heights[-1].

If the element in question has no left child then left_child_index should be nil. To ensure that we catch this scenario:

left_child_index = nil if tree[2*i + 1].nil? (S3.3)

Putting S3.2 and S3.3 together using a ternary:

left_child_index = tree[2*i + 1].nil? ? nil : heights.length - i -1 (S3.4)

Therefore, the height of the left child will have to be 0 if left child is nil. The full formula for left_child_height then is:

left_child_height = left_child_index.nil? ? 0 : heights[left_child_index] (S3.5)

Step 4: Find height of right child — finding the height of the right child of a sub tree follows the same logic as Step 3. Since we are filling up heights array from left to right (using push) and we are iterating tree from right to left, the height of the right child of any sub tree will always be pushed first to heights. Therefore, the left child of any element will be at position left_child_index -1 inside heights (if right child is not nil in tree). Taking these into consideration and following the logic of Step 3:

right_child_index = tree[2*i + 2].nil? nil : left_child_index - 1 (S4.1)

right_child_height = right_child_index.nil? ? 0 : heights[right_child_index] (S4.2)

Step 5: Find element’s total height — After finding the heights of the left and right children of the element in question (at i index in Ltree), we can now find that element’s total height:

total_height = initial_height + [left_child_height, right_child_height].max (S5.1)

Numerically speaking, if the element is 0 and it happens to have any child(ren) inside tree then such child(ren) will also be 0. Hence, its total_height will also be 0. Such is the case with element at i = 5 in T0 above:

                                         left  right
                                         child child
tree = [1, 7, 5, 2, 6, 0,  9, 3, 7, 5, 11, 0,   0,   4, 0] 
                      i=5                i=11 i=12
                  element in question
(T0 here repeated)
total_height = 0 + [0,0].max = 0 (S5.2)

But for the element at i = 4, the height is:

                                    left   right
                                    child  child
tree = [1, 7, 5, 2, 6, 0,  9, 3, 7,   5,    11,     0, 0, 4, 0] 
                   i=4               i=9  i=10
                  element 
                 in question
total_height = 1 + [1,1].max = 2 (S5.3)

In S5.3 and S5.4 above we just used visual inspection to compute the heights of the right and left children of the element in question. But this illustrates how our algorithm works. Now after computing for the total_height we simply:

Step 6: Push total_height into heights — As I noted before, using the push method is more efficient, especially for large arrays.

heights.push(total_height) (S6.1)

Once we have iterated through all elements in the tree array, we will have an array heights composed of the heights of each sub tree in the binary tree. It should look like this:

heights(after full iteration) = [0, 1, 0, 0, 1, 1, 1, 1, 2, 0, 2, 2, 3, 3, 4] (S6.2)

Step 7: Return height of the binary tree — If our goal is just find out the height of the mother tree (meaning from the root down to the lowest-rightmost node) then we simply:

return heights[-1] (S7.1)
*Note if this is the last line in the method then the 'return' keyword is redundant (in Ruby at least)

However, a lot of times we may be interested to compute for the heights of any of the sub trees. In that case we simply return the heights array itself and then anyone using the program can simply include any index to find the height of a specific branch in the tree.

The full method below:

def binary_tree_height(tree_array)
  #0 Declare a heights array which will store the heights of each sub tree
  heights = []
  #1 Iterate through the tree_array starting from last element down to first
  (tree_array.length - 1).downto(0) do |i|

  #2 For each element, find initial height
  initial_height = tree_array[i] == 0 ? 0 : 1

  # 3 Find height of left child
  left_child_index = tree_array[2*i + 1].nil? ? nil : heights.length - i - 1 #index of left child's height in heights
  left_child_height = left_child_index.nil? ? 0 : heights[left_child_index] 

  # 4 Find height of right child
  right_child_index = tree_array[2*i + 2].nil? ? nil : left_child_index - 1 #index of right child's height in heights
  right_child_height = right_child_index.nil? ? 0 : heights[right_child_index]

  # 5 Find element's total height
  total_height = initial_height + [left_child_height,right_child_height].max

  # 6 Push total height to heights array
  heights.push(total_height)

 end
 puts heights[-1]
end

Let’s test this algorithm out.

Let us suppose we run binary_tree_height(tree). Computing for the heights of tree[14] down to tree[7] is pretty straightforward (they will either be 0 or 1 since they are all at the lowest level of tree) so we won’t simulate them anymore here. We will assume we are already in that part of the iteration when i will be equal to 6. Therefore, at this juncture:

i = 6 (F1)
tree[6] = 9 (F2)
heights = [0, 1, 0, 0, 1, 1, 1, 1] (heights.length at this point is 8) (F3)

Now, we can see that tree[6] is equal to 9 (and not 0). Therefore:

initial_height = 1 (F4)

As promised, here is how I came up with the formula for the indices of the left and right children.

So I began with a heights array already filled with the heights of the lowest elements as shown in F3. Since I’m now working with tree[6] (which is 9) then its left and right children are tree[13] and tree[14]; whose corresponding heights are in heights[1] and heights[0], respectively. If that’s not clear enough, we know we push starting from tree[14] — this will become heights[0]. We then compute for and push the height of tree[13] — this will be heights[1]. Relating the indices:

index of left child in trees = 13
index of left child's height in heights = LEFT_INDEX =1
index of right child in trees = 14
index of right child's height in heights = RIGHT_INDEX = 0
current index of element in question = MOTHER_INDEX = 6
current length of heights array = LENGTH = 8
LEFT_INDEX = 1 = 8 - 6 - 1 = LENGTH - MOTHER_INDEX - 1
RIGHT_INDEX = 0 = 8 - 6 - 2 = LENGTH - MOTHER_INDEX - 2 
(or simply LEFT_INDEX -1 ) (F5)

We can now apply this logic to all elements, so then in code we compute for the height of tree[6] as follows:

Computing for tree[6]'s left child's height:
from code at S3.4:
left_child_index = tree[2*i + 1].nil? ? nil : heights.length - i - 1
Since tree[2*6 + 1] = tree[13] = 4 is not nil then:
left_child_index = 8 - 6 - 1 = 1
from code at S3.5:
left_child_height = left_child_index.nil? ? 0 : heights[left_child_index]
So then:
left_child_height = heights[1] = 1

Following the same for tree[6]’s right child’s height:

from code at S4.1:
right_child_index = tree[2*i + 2].nil? nil : left_child_index - 1 
Since tree[2*6 + 2] = tree[14] = 4 and is not nil:
right_child_index = left_child_index -1 = 1 -1 = 0 -> !nil?
and from code at S4.2:
right_child_height = right_child_index.nil? ? 0 : heights[right_child_index]
Therefore: right_child_height = heights[0] = 0

Now we can find the total height of tree[6]:

total_height (tree[6]) = 1 + [1,0].max = 1 + 1 = 2

We can then push this total_height into heights:

heights.push(2), such that:

heights = [0, 1, 0, 0, 1, 1, 1, 1, 2]

And the same thing goes on until we work on tree[0] and the final heights array should be:

heights = [0, 1, 0, 0, 1, 1, 1, 1, 2, 0, 2, 2, 3, 3, 4]

And returning heights[-1] (or heights[heights.length -1], whichever we prefer), we determine that the height of tree is 4. We can verify this visually in both figures 1 and 2 above.

It took us 7 steps to come up with the answer. With this size of tree array the operation took around 0.024 milliseconds to finish. It takes half the time (only 0.012 milliseconds) for the same thing to be accomplished using recursion.

As a preview on how to do this recursively, we can simply do something like:

def tree_height_recursive(tree_array, index = 0)
  return 0 if tree_array[index].nil? or tree_array[index] == 0
  left_child_height = recursive_tree_height(tree_array, 2*index + 1)
  right_child_height = recursive_tree_height(tree_array, 2*index +2)
  total_height = 1 + [left_child_height, right_child_height].max
end

We see that recursion probably will only take us at most 4 steps to do the same task. And it saves us half of the time and less resources used.

One secret for learning algorithms is hard work and practice. It also helps if you work collaboratively with others. I actually did the above not alone but with my coding partner. I previously wrote about how learning this way is so much more productive and effective.

Here is my repository on the different data structures and algorithms that I’ve worked on.

Follow me on Twitter | Github