And that's great – but you should be careful that you don't end up using a logical fallacy to help you make your point.

What is a Logical Fallacy?

A logical fallacy is an error in reasoning that makes your argument less effective and convincing. And you want to be able to spot these fallacies in other people's arguments (and your own) so you can call them out or fix your own strategy.

There are two major types of logical fallacies, formal and informal.

In formal fallacies, there's a problem with how you structure your argument, and how you're making your points. You might be speaking the truth, but the logic breaks down because of the way you're putting your arguments together.

In informal fallacies, there's a problem with what you're saying, and the information might be incorrect or misleading.

In this article, we'll focus on these informal fallacies as they can be pretty common in everyday debate. And keep in mind that we're not talking about the effectiveness or persuasiveness of your argument, here – after all, fallacious arguments can be very persuasive.

Instead, it's all about giving you the tools to identify these weak arguments so you don't make these mistakes in your reasoning.

List of Logical Fallacies with Examples

In this article, we'll look at the most common informal fallacies so you can learn to identify them and avoid them.

The Sunk Cost Fallacy – Definition and Example

Have you ever finished a task (that you really didn't want to complete) simply because you'd put so much time and effort in already? You probably felt like you didn't want all that hard work to go to waste, or to be for nothing.

You were likely falling prey to the sunk cost fallacy. It states that it's actually better to abandon a project that's going nowhere (at any point) rather than waste any more time, energy, and resources trying to finish it for the sole purpose of finishing it.

The reason for this might seem counterintuitive, but think about it: rather than spend another minute of your precious time doing something that isn't going anywhere, it's better to switch gears ASAP (before you spend any more time) and start putting your energy into something productive.

Example of a Sunk Cost Fallacy

Let's say that you've decided to write a book. You spend hours and hours doing research, making an outline, and writing the first 10 chapters. You've put months if not years of your life into writing this book.

But then perhaps your interests change, or you no longer wish to be an author. You might think you should finish the book because you're so close or because you've already spent so much time and energy on it.

Instead, though, you should leave that project behind and focus on what's ahead. Maybe you're trying to get a new job, or learn a new skill, or move to a new city. Any of these current and relevant initiatives would suffer if you continued to work on your unsuccessful book project.

So how do you distinguish between this sunk cost fallacy and persevering until you finish something difficult? Well, it helps to think about whether the experience will benefit you in the long run – in which case, it would be helpful to see it through.

For example, let's say you've done three years of a four year degree program at a college or university. But your interests have changed, and you want to pursue something that doesn't require that degree.

Still, it might make sense to finish the program, as a college degree typically only helps you in future career moves – not to mention the life experience you'll gain in the process.

The Ad Hominem Fallacy – Definition and Example

Ad hominem means "against the person" in Latin. So the ad hominem fallacy happens when you attack a person's character, appearance, personality, or other irrelevant aspects in an argument instead of attacking what they're saying.

These types of attacks are fallacious because they're not relevant to the argument, and so they distract from the point at hand. It doesn't really matter if you think your mom is being a jerk – she's still right that you shouldn't speed while driving.

Example of an Ad Hominem Fallacy

Many people associate ad hominem fallacies with political debates. Unfortunately, some candidates don't seem to be able to help themselves.

What if Candidate A said that you shouldn't trust Candidate B because Candidate B doesn't dress well? There's no established link (that I know of!) connecting a "good dresser" with trustworthiness or good political decision-making, so this would be an ad hominem fallacy.

Or what about when Candidate A insults Candidate B for being too nerdy, or not cool enough? These qualities, first of all, are subjective, and second, they shouldn't affect Candidate B's ability to govern effectively.

On the other hand, sometimes people just deliver insults that aren't actually logical fallacies because they aren't part of the argument. For example, if you were to say that all New Yorkers are rude and unfriendly (but you aren't trying to make a point), that's just an (untrue) insult and not a fallacy.

So when you're debating someone, leave their personal characteristics out of it unless they're relevant to your point.

The Straw Man Fallacy – Definition and Example

When you hear the term "straw man", what comes to mind? Probably a figure of a person made of straw, like a scarecrow, or something else insubstantial. That straw figure isn't too solid, and you could just knock it over with a little push or a strong gust of wind.

The same holds true for straw man fallacies – they represent weaker arguments that are oversimplified or that distract from the main point the debater is trying to make.

So instead of responding to someone with a well-reasoned, to-the-point counterargument, someone using a straw man might reframe that person's argument in a vastly oversimplified way, or might latch on to an irrelevant point that's tangentially related and go after that. Basically, they create a "straw man" in place of a real argument.

Example of a Straw Man Fallacy

Perhaps you're discussing education with someone who believes that for-profit colleges are harmful to the broader educational system because they take advantage of their students, don't provide them high-quality education, and waste students' money.

Instead of responding with appropriate counterpoints (such as concrete examples of for-profit colleges who benefit their students), you try to undermine the person's argument by saying "See, they're against higher education and don't think people should go to college!"

In fact, the person has a much more nuanced claim, but you've ignored it and constructed a vague straw man fallacy in response.

Or maybe you're trying to figure out a solution to the number of people living without homes in your area. You might suggest setting up temporary (or permanent) tiny homes for houseless individuals, allocating resources for trash cleanup, and providing medical care during the pandemic.

Your opponent, however, might misconstrue your argument and insist that you're trying to welcome the homeless community to your area by providing so many benefits for them.

The False Dilemma Fallacy (AKA The False Dichotomy Fallacy) – Definition and Example

Have you ever argued with someone and they only give you two options when you feel like there are many more? Chances are they were falling into the trap of the false dichotomy.

Using a false dichotomy or false dilemma in an argument means that you oversimplify your argument or only focus on two outcomes when in fact there are other reasonable possibilities.

This strategy tries to hide important facts and considerations and tries to trick your opponent into thinking the argument is more cut and dry or simpler than it really is.

Example of a False Dilemma Fallacy

Let's say that you're still working on finding homes for houseless people in your community. You might suggest a range of housing options, such as tiny houses, community living, repurposing empty apartment buildings, and so on.

You could also offer to relocate people who wished to leave your area, or you could help them find jobs so they could afford their own home eventually.

Someone opposed to your efforts might say that houseless people either need to get a job so they can afford their own place or leave town. And they wouldn't offer any of the other options you explored.

To someone uninformed about the crisis of homelessness in your area, those two options might sound reasonable. But to someone who had studied the issue extensively, it would be clear that those extremes weren't the only options.

How about another example?

Maybe you're at a political debate and one of the candidates asserts that you're either a Democrat or you're a Republican in an effort to make some point.

In reality, though, this likely wouldn't be the case. Certain people in attendance could be Libertarians, for example – but the politician didn't include that as an option.

So keep in mind, when you're making an argument, that there are likely many nuances that relate to your point. Don't ignore them – simply take them into account and build them into your argument.

Do keep in mind, though, that some arguments really only do have two viable options – so they wouldn't represent false dichotomies. For example, if a General says "Either you're with us or you're against us" during a war, those are the two main options.

The Slippery Slope Fallacy – Definition and Example

The slippery slope fallacy refers to arguments that get increasingly dramatic and out of hand very quickly. Especially when the ever-more-dramatic conclusions aren't realistic or likely to happen.

These types of arguments are often made when someone wants to emphasize how drastically bad an outcome would be.

Perhaps a better name for this fallacy, though, would be the Domino Effect – one thing might lead to another which might lead to another which might...and so on. The problem with these assumptions is that they're all hypothetical, which makes your overall claim very weak.

Example of a Slippery Slope Fallacy

Perhaps your teenager wants to buy themselves a truck. They've been saving up, and they have the money. But you don't want them to drive a truck, for any number of reasons – perhaps you're worried about gas mileage, or parking in a city, or that they'll take it off-roading and get hurt.

Now, these are all fairly reasonable arguments as to why you wouldn't want your kid driving a truck, and they could easily result from that purchase.

But what if, instead of these sensible arguments, you let your emotions get away with you and instead said "You can't get a truck because then all your friends will want trucks and their whole families will then get trucks which they'll start driving all over the place and over-polluting the earth!"

You can see how that escalated quickly, right? And even though the arguer has a point about emissions in general here, it's probably not a realistic outcome of this situation (and it's probably not an effective argument to use to convince your teen not to buy a truck).

The Circular Reasoning Fallacy – Definition and Example

Have you ever noticed someone arguing in a way that they seem to go around in a circle? It might seem like they're making an argument, but they'll use their conclusion to justify their argument, and their argument to justify their conclusion.

If this sounds confusing, that's because it is. When someone says something like "This tee-shirt is wet because it's covered in water," they're making a fallacious argument. In fact, the tee-shirt is wet because you fell in a lake, for example.

In this case, someone saying something's wet because it's covered in water is just stating the obvious. They're not offering an explanation for why it's that way.

You can often recognize a circular argument when the conclusion – the thing the person is arguing in favor of (or against) – is also one of the premises (or arguments) they're using to justify their assertion (it's wet because of water, which is wet). In other words, if this is true because that is true, that is true because this is true.

Example of a Circular Reasoning Fallacy

So here's another example: you say that your friend Jessie lies all the time, and you know this because they never tell the truth. But your argument (that Jessie lies all the time) and your premise (because they never tell the truth) are the same thing. That means that this is a circular argument.

Here's another way to think about it: if your argument's premises assume that your conclusion is true right from the beginning, rather than proving or finding that it's true, you're arguing in a circle. Just remember: if your argument is defined in terms of itself, it is probably fallacious.

And if you want to know why it's sometimes called "Begging the Question," you can read all about it here. (Hint: it's a mistranslation of 16th century Latin that was actually a mistranslation of the ancient Greek phrase...fascinating.)

The Equivocation Fallacy – Definition and Example

Equivocation means that you're taking a word or phrase and changing its meaning slightly so that it means something else. Or you're using one word or phrase instead of another to hide the true meaning of what you're saying.

In other words, you're being ambiguous with your language. If something is ambiguous, it means that you can interpret it in more than one way or that it has two meanings. This is exactly what happens in an equivocation fallacy.

The word "equivocation" comes from the Latin for "equal voice" – meaning that it appears that what you're saying means one thing but it really means or can also mean something else.

The important thing to remember about equivocation fallacies is that they attempt to deceive in some way.

You might jokingly use ambiguity in a story, play, or playful conversation – but you're not really trying to convince your listener of something serious (or it's clear that you're being tricky or silly).

But when you use equivocation in a serious debate, political campaign, advertisement, or something similar, that's when it's more malicious and fallacious.

Example of an Equivocation Fallacy

So how do you tell the difference? Be mindful of the setting in which you use ambiguous language, or you see it being used.

Here's a simple example: "Nine out of ten dentists recommend Colgate toothpaste." First of all, what does "recommend" mean here? This could be misleading – do they really specifically recommend Colgate, or do they just recommend that you brush your teeth in general?

How about another example? What if you break up with someone, and they ask you never to drive by their house again. So you walk by – but you justify it by saying that you didn't drive by. You walked.

Clearly your ex meant that they didn't want you going by their house in any way, but you used the ambiguity of the situation to tweak their words and do it anyway.

The Post Hoc Fallacy – Definition and Example

You might have heard the phrase "post hoc ergo propter hoc" before, even if you've never studied Latin.

This Latin phrase translates to "After this, therefore because of this." Now that might sound like a jumble of conjunctions and such, but it basically means that if event B happened after event A, that must mean that event A caused event B.

Post hoc ergo propter hoc → (B is) After this (A), therefore (B is) because of this (A).

This fallacy says that because one thing happened after another, it means that the first thing caused the second thing happen. The argument is a fallacy when someone asserts something based purely on the order that things happened. This means they're not taking into account other factors that affected or caused the event to happen.

If this sounds a bit familiar to you, it means you might have thought about correlation vs causation before. The post hoc fallacy is related, but is more focused on the order of events (and their relationship).

Example of a Post Hoc Fallacy

Let's look at an example to help decipher what's going on in this type of fallacious argument.

Maybe there was an earthquake during which a building fell down. That's a pretty clear example of causality – the earthquake (event A) caused the building to fall down (event B).

But what if, after that same earthquake, a lot of people moved away from the city? Now, some of them might have moved because the earthquake was the last straw. But many might have fled because of rising housing costs, pollution, over-crowding, poor infrastructure, poor schools, or a bunch of other factors.

In other words, the earthquake likely wasn't the only direct cause of people moving away.

So anyone who argued "Look, people are moving out of the city because of the earthquake!" and didn't account for all these other likely causes was making a fallacious argument.

Here's another example: perhaps you're searching for a job, and you're not having any luck. But then someone gives you a good luck charm, and after a few more applications, you get a job.

You might be tempted to think that the good luck charm got you the job. But what's more probable is that you put a lot of effort into your applications, you studied really hard for your interviews, and you found your perfect company fit.

The Appeal to Authority Fallacy (AKA Argumentum ad Verecundium Fallacy) – Definition and Example

When you're gathering evidence to support your conclusion, you'll likely want to cite some experts. They've done research on the subject and know a lot about it, so it makes sense to use their knowledge and opinions to support your own arguments.

But be careful – if you don't use those expert's information correctly, or if you assume they're always right because they're experts, you could be falling prey to the appeal to authority fallacy.

An appeal to authority fallacy is easy to commit, but can be hard to recognize. This is because of the weight we all give to "authorities" in various subjects.

When you're engaging in an appeal to authority fallacy, you're likely either misusing someone's authority, citing an irrelevant authority, or citing a poor authority.

Let's see what these look like with some examples.

Example of an Appeal to Authority Fallacy

Let's say your mom's a lawyer and you seek her advice about a particular legal problem you have. If she practices that type of law and has experience with the problem you're having, you can likely cite her authoritative opinion with confidence.

But if you're arguing with your mom about the best way to save the sea turtles, and she asserts that she knows best because she's an intelligent person, she's using her own authority in a fallacious way (and with little to no justification).

Here's another example. Perhaps you watch a lot of Greenbay Packers football, and Aaron Rogers is your favorite quarterback. You happen to see a State Farm insurance commercial where Aaron endorses State Farm's services. You might think, "Well, I like Aaron Rogers, and he recommends State Farm, so it must be great insurance!"

While State Farm might be great insurance, Aaron Rogers doesn't have the authority to say so. He's an authority on being a great quarterback, but not on the quality or efficacy of insurance. So this is an example of an irrelevant appeal to authority.

So, when you're searching for evidence to back your claim, just remember – authorities aren't the only sources you should cite.

And you shouldn't just expect people to trust what those experts say with no evidence. After all, even the experts can be wrong, and just because they know a lot about one thing doesn't mean they know a lot about everything.

The Appeal to Ignorance Fallacy – Definition and Example

No one knows everything – it's just a fact of being human. We're all still learning, and while some might know more than others, we'll all be ignorant about certain things.

With that in mind, it's pretty easy to see why the appeal to ignorance fallacy is so common and so useless.

When you say something like "Well, no one's ever seen Nessie (the Loch Ness Monster) before, so they can't prove that she's real", you're making an appeal to ignorance. Why? Because no one knows whether she exists or not – because they've never seen her!

But the clearest way you can tell this is an appeal to ignorance fallacy is that you can turn it right around, and it still seems to make sense: "Well, no one's ever seen Nessie before, so they can't prove that she's not real!"

Either way, in both these claims, you're making an assertion based on something no one knows (the ignorance bit). Because no one knows it, you shouldn't use it in an argument.

Example of an Appeal to Ignorance Fallacy

Let's look at another example of an appeal to ignorance fallacy in action.

Perhaps you're an archaeologist who's studying an ancient civilization that lived around 2000 years ago. You study any remaining stone structures, pottery, tools, jewelry, and anything else they left behind.

You try to piece together what life would've looked like for these people based on their artifacts, where they lived, nearby societies, and so on. But you have no written evidence that tells you anything more. No one has found any inscriptions, written documents, or anything else with writing on it.

It would be tempting to assert that, since no one has ever found any evidence of writing, this society didn't have a written language. "We've never found documents or inscriptions, so they must not have written their language down."

But you could also assert that, even though no one has found those documents yet, they still might be out there and just haven't been excavated and discovered yet.

This argument is an appeal to ignorance, because you don't know something/haven't seen any evidence of something, but you're using it to support your argument (that the society doesn't have a written language) all the same.

The Appeal to Popular Opinion Fallacy (AKA Bandwagon Fallacy or Ad Populum Fallacy) – Definition and Example

Have you ever heard the expression "jumping on the bandwagon"? It refers to someone changing their opinion or developing an opinion just because a bunch of people hold that same opinion.

There's not necessarily good evidence for that opinion, but people hold it anyway – maybe because it's been believed for a long time, or just because of the sheer number of people who believe it. But even though many people believe this thing, it may be factually incorrect or misleading.

This is a form of the appeal to popular opinion fallacy. You argue that something is true, good, or right just because a large number of people (or some popular or influential person or people) are doing it or believe it.

What's wrong with that? If everybody's doing it, it must be good – right? Well, not necessarily. People aren't always completely rational and don't always think things through. Think of the term "mob mentality". What does that conjure up? Probably a bunch of people causing chaos – in other words, not a good thing.

So before you say something like "Well everyone believes this, so it must be true", think again. Because this isn't a case of "strength in numbers" – an ad populum fallacy results from a lot of people believing incorrect or misleading information.

Example of an Appeal to Popular Opinion Fallacy

What if your young teenager comes to you and wants to get a tattoo. They argue that all their high school friends are doing it because some celebrity just got this new tattoo.

Now, whatever your feelings about tattoos, this is a logical fallacy. Just because everyone's getting this tattoo doesn't mean it's the right choice for your kid. Maybe they haven't thought it through, or maybe they can't handle serious pain/needles, or maybe they will change their mind in a few years and regret such a permanent choice.

Also, everyone has different reasons for getting tattoos. Some do it to commemorate someone or something, some do it for the beauty of the art, some do it while intoxicated on vacation, and so on. But if a group of young teenagers is getting a tattoo on a whim to copy a celebrity, perhaps that's something you want your kid to think about more carefully.

So your kid arguing that "all my friends are doing it, so it's cool" doesn't take that into account. They'd need to think about getting a tattoo for their own reasons, and justify it to you that way.

Here's another example: you're FaceTiming with your family, and it's an election year. Most of your family belongs to one political party, but you belong to another.

Your mom starts trying to convince you to vote like they do – "The whole family votes this way! And we've been voting this way forever! Come on, you should be like your family and support the same candidate/things we do."

While it's understandable that your mom would want your political beliefs to align with hers, she's making a fallacious argument here. Just because they've always voted that way doesn't make it right.

She shouldn't say you should vote like she does because "that's what the family's always done/it's what they all do now". She should point out the benefits of her candidate, how they could help you out, why their policies are fair, and so on – and then let you decide for yourself.

The Hasty Generalization Fallacy – Definition and Example

People make generalizations all the time (that, right there, was a generalization!). And sometimes this is ok. If you're just stating something that's generally true, like "I like to cook" or "Puppies are cute", there's typically no harm in that.

The problem arises, though, when someone uses a generalization a bit too zealously in an argument without sufficient evidence. These types of "hasty" generalizations can fall into stereotyping, racism, falsehood, exaggeration, and more.

Often someone makes such a generalization when they're basing their opinion or argument off of the behavior or characteristics of just a few members of a group. This often means they're not taking the behavior of the whole group into consideration.

So why are these generalizations bad? Aside from lacking evidence and being based on problematic premises, people often assert hasty generalizations as if they were 100% true all the time. Which, of course, very few likely are.

If you want to avoid making hasty generalizations, you can use certain qualifiers when you make a generalization – like "Sometimes", "Often", "We often see", or "It may be the case that...". Those types of words and phrases let your listener know that you're not arguing that this thing is true across the board for everyone. It's just a general trend you've noticed.

Example of a Hasty Generalization Fallacy

Hasty generalizations are quite common, as people use generalizations all the time in regular conversation. And again, many generalizations don't hurt anyone. But let's look at some examples of bad generalizations.

If you say "People in the southern part of the US are so conservative and close-minded. I really can't stand how all they care about is football and BBQ", you're using a hasty generalization (a couple, actually).

While it's true that some people in the south have these characteristics, it's not true for everyone living in that region. And by making those assertions, you're perpetuating stereotypes that are likely overblown and miss a lot of nuance about southern American's characters and beliefs.

Here's another example: let's say you're having a fight with your significant other and you say, "You always pick fights with me!", you're likely exaggerating and making a hasty generalization. Unless it's literally true that they are always the one to start the fight, you're probably getting carried away in the heat of the moment.

One way to save yourself from making a hasty generalization in this case would be to say something like "You pick fights with me a lot" or "You often pick fights with me."

The Tu Quoque Fallacy (AKA Appeal to Hypocrisy Fallacy) – Definition and Example

Tu quoque in Latin means "You, too". And when you attempt to distract from your own guilt by calling out someone else's similar guilt, you're committing this fallacy.

The name makes sense – it's like you're saying "Well I may have done this, but you did it, too!" Now, think about that. Just because someone else did something similar to (or the same as) what you did, it doesn't make you any less guilty. You've still committed whatever crime or done whatever bad thing you've done.

This is also called an "appeal to hypocrisy" fallacy, because the person making the argument (let's call them Person A) often calls out the fact that someone else (Person B) did something similar to what they did. Person A argues that they may have messed up, but Person B did the same thing so should be punished. Person A is being a hypocrite because they're trying to escape the blame they'd like to assign to Person B.

It's tempting to use this type of argument, because people are always looking to shift the blame from themselves to others. It's especially enticing when that other person is not blameless and therefore seems to deserve some share of the guilt.

But this isn't an effective argument strategy because, while distracting, a tu quoque argument doesn't actually prove you innocent. It just draws attention (falsely) away from the issue at hand, which is your misdeed.

One thing to remember about tu quoque fallacies is that the information the person making the argument cites is typically irrelevant to the case at hand. Just because Person B is guilty also, doesn't mean Person A is any less guilty. So that accusation that Person A makes is irrelevant to their case.

Example of a Tu Quoque Fallacy

Let's go back to our teenager. Perhaps they've been caught skipping school, and their parents want to ground them for a week. The teenager might argue, "Yeah I skipped third and fourth periods, but Marta did, too!"

While it's not great that Marta skipped class as well, it doesn't really make that teen any less guilty of skipping school. They just knew someone who did the same thing, and are trying to justify what they did by bringing up Marta's transgression as well. But it doesn't mean that they skipped any less school.

Here's another example: perhaps your friend caught you cheating on a test, and threatened to turn you into the teacher. But you saw them cheat in another class last year, so you say "I may have cheated today, but you cheated on that math test last year, too!"

Again, their cheating a year ago doesn't make you any less guilty right now. While it might feel good to say, "You did that, too, so how could you think I should be punished for it!", it's not really a strong or relevant argument to make.

Instead of resorting to this type of argument, make sure you take responsibility for your actions and keep your points relevant to the issue at hand. Don't think you can get away with something just by calling out someone else's hypocrisy. It's likely not going to help your case.

The Loaded Question Fallacy – Example and Definition

When you ask a question that intends to reinforce your position and undermine someone else's, you could be asking a loaded question. These questions are helpful to you but harmful to the person you're asking, and may skew the opinion of anyone listening in your favor, perhaps unfairly.

Instead of asking a straightforward question that attempts to get more or new information, a loaded question often includes an accusation (or a confirmation of an accusation) – an oft-quoted example is "Are you still beating your wife?"

In this question, you're referencing an accusation – that the person beat their wife – without directly accusing them of doing it currently. But by including it in the question, you're turning listeners' minds to the fact that this person did, at one point, beat their wife. So either way, they'll appear guilty.

Example of a Loaded Question Fallacy

Let's look at some more examples of loaded questions, and why they're fallacies.

Perhaps you're at a rally in support of clean energy, and a rep from Exxon is there. If you're not old enough to remember, Exxon had a horrific oil spill in Alaska in 1989 that devastated 1300 miles of coastline and released over 10 million gallons of oil into the ocean.

You might call out that rep and loudly ask them if their company is still polluting the world's pristine oceans and killing millions of sea creatures.

Whatever your feelings about Exxon or environmental justice, it's not fair to set the company up like that for those listening. Your question is heavily loaded, and doesn't give them a shot at convincing others of their current position, whatever it might be. You're making your argument by essentially biasing the crowd against them from the start.

Here's another example: what if a company hires formerly incarcerated people, and you find out that one of them was a bank robber. If you asked their employer "You're really gonna let a thief handle your products?" you're creating a negative bias against them.

It's not necessary to refer to them as a thief or allude to their past as a bank robber. By doing so, you're only creating prejudicial feelings against them that may not be relevant or meaningful at this point in time.

So just remember – when you're asking questions to try to prove your point, keep them relevant, unbiased, and focused on the issue at hand.

The Red Herring Fallacy – Definition and Example

You might wonder where the term "red herring" comes from. It's a bit of an odd name for a fallacy, don't you think?

Well, there has been some debate about this in the past but most sources agree that a red herring signifies a distraction or something meant to mislead someone.

Fun fact before we continue: there's not actually a species of herring called a red herring. A "red herring" refers to a herring that's been brined and smoked until it becomes extremely pungent and turns a bright red color.

So these red herrings were used as training aids for animals because of their strong smell (to attempt to lead them in a certain direction).

Anyway, back to our fallacy: if you make an argument with the intention of distracting from the real issue at hand, it might be a red herring. Also, if you drop some seemingly related bit of info into a conversation or debate that leads your listener down the wrong path, that's also a red herring.

Ultimately, a red herring argument distracts or leads your listener away from the crux of the issue so that they get off course or off topic.

Example of a Red Herring Fallacy

Remember, a red herring basically a diversionary tactic in an argument. It's meant to lead the listener away from the main point of the conversation.

Suppose you're arguing with someone who is in favor of a dam that's being constructed in a beautiful river. You bring up the environmental impact that said dam will have, and how devastating it'll be to the surrounding natural habitat.

Your opponent might say something like "Yes it will destroy the habitat for many fish and other river animals, but if we don't build the dam it'll take jobs away from so many people who would've worked on it."

Now, this person has just used a red herring fallacy to try to distract from the environmental impact of such a dam. Instead of arguing for the benefits of the dam itself, and arguing against the environmental impact, they're dropping in a red herring – the potential impact on the workers who would've been hired to build the dam.

While that itself is a whole separate issue, it doesn't deal with or respond to the issue at hand, which is what happens to the natural environment when the dam goes in.

How to Avoid Logical Fallacies in Your Arguments

We've just discussed a whole bunch of logical fallacies, and you might be thinking – how can I make any arguments at all without saying something fallacious?

It's not always easy, as some of these fallacies are very tempting and easy to fall into. But as long as you stick to the point, don't try to deceive your listener, cite relevant evidence from relevant sources, and avoid any derogatory or misleading language, you should be ok.

Good luck, and happy debating!

Two of those methods are inductive and deductive reasoning.

So what's the difference between inductive and deductive reasoning, when should you use each method, and is one better than the other?

We'll answer those questions and give you some examples of both types of reasoning in this article.

What is Inductive Reasoning?

The method behind inductive reasoning

When you're using inductive reasoning to conduct research, you're basing your conclusions off your observations. You gather information - from talking to people, reading old newspapers, observing people, animals, or objects in their natural habitat, and so on.

Inductive reasoning helps you take these observations and form them into a theory. So you're starting with some more specific information (what you've seen/heard) and you're using it to form a more general theory about the way things are.

What does the inductive reasoning process look like?

You can think of this process as a reverse funnel – starting with more specifics and getting broader as you reach your conclusions (theory).

Some people like to think of it as a "bottom up" approach (meaning you're starting at the bottom with the info and are going up to the top where the theory forms).

Here's an example of an inductive argument:

Observation (premise): My Welsh Corgis were incredibly stubborn and independent (specific observation of behavior).
Observation (premise): My neighbor's Corgis are the same way (another specific observation of behavior).
Theory: All Welsh Corgis are incredibly stubborn and independent (general statement about the behavior of Corgis).

As you can see, I'm basing my theory on my observations of the behavior of a number of Corgis. Since I only have a small amount of data, my conclusion or theory will be quite weak.

If I was able to observe the behavior of 1000 Corgis (omg that would be amazing), my conclusion would be stronger – but still not certain. Because what if 10 of them were extremely well-behaved and obedient? Or what if the 1001st Corgi was?

So, as you can see, I can make a general statement about Corgis being stubborn, but I can't say that ALL of them are.

What can you conclude with inductive reasoning?

As I just discussed, one of the main things to know about inductive reasoning is that any conclusions you make from inductive research will not be 100% certain or confirmed.

Let's talk about the language we use to describe inductive arguments and conclusions. You can have a strong argument (if your premise(s) are true, meaning your conclusion is probably true). And that argument becomes cogent if the conclusion ends up being true.

Still, even if the premises of your argument are true, and that means that your conclusion is probably true, or likely true, or true much of the time – it's not certain.

And – weirdly enough – your conclusion can still be false even if all your premises are true (my Corgis were stubborn, my neighbor's corgis were stubborn, perhaps a friend's Corgis and the Queen of England's Corgis were stubborn...but that doesn't guarantee that all Corgis are stubborn).

How to make your inductive arguments stronger

If you want to make sure your inductive arguments are as strong as possible, there are a couple things you can do.

First of all, make sure you have a large data set to work with. The larger your sample size, the stronger (and more certain/conclusive) your results will be. Again, thousands of Corgis are better than four (I mean, always, amiright?).

Second, make sure you're taking a random and representative sample of the population you're studying. So, for example, don't just study Corgi puppies (cute as they may be). Or show Corgis (theoretically they're better trained). You'd want to make sure you looked at Corgis from all walks of life and of all ages.

If you want to dig deeper into inductive reasoning, look into the three different types – generalization, analogy, and causal inference. You can also look into the two main methods of inductive reasoning, enumerative and eliminative. But those things are a bit out of the scope of this beginner's guide. :)

What is Deductive Reasoning?

The method behind deductive reasoning

In order to use deductive reasoning, you have to have a theory to begin with. So inductive reasoning usually comes before deductive in your research process.

Once you have a theory, you'll want to test it to see if it's valid and your conclusions are sound. You do this by performing experiments and testing your theory, narrowing down your ideas as the results come in. You perform these tests until only valid conclusions remain.

What does the deductive reasoning process look like?

You can think of this as a proper funnel – you start with the broad open top end of the funnel and get more specific and narrower as you conduct your deductive research.

Some people like to think of this as a "top down" approach (meaning you're starting at the top with your theory, and are working your way down to the bottom/specifics). I think it helps to think of this as "reductive" reasoning – you're reducing your theories and hypotheses down into certain conclusions.

Here's an example of a deductive argument:

We'll use a classic example of deductive reasoning here – because I used to study Greek Archaeology, history, and language:

Theory: All men are mortal
Premise: Socrates is a man
Conclusion: Therefore, Socrates is mortal

As you can see here, we start off with a general theory – that all men are mortal. (This is assuming you don't believe in elves, fairies, and other beings...)

Then we make an observation (develop a premise) about a particular example of our data set (Socrates). That is, we say that he is a man, which we can establish as a fact.

Finally, because Socrates is a man, and based on our theory, we conclude that Socrates is therefore mortal (since all men are mortal, and he's a man).

You'll notice that deductive reasoning relies less on information that could be biased or uncertain. It uses facts to prove the theory you're trying to prove. If any of your facts lead to false premises, then the conclusion is invalid. And you start the process over.

What can you conclude with deductive reasoning?

Deductive reasoning gives you a certain and conclusive answer to your original question or theory. A deductive argument is only valid if the premises are true. And the arguments are sound when the conclusion, following those valid arguments, is true.

To me, this sounds a bit more like the scientific method. You have a theory, test that theory, and then confirm it with conclusive/valid results.

To boil it all down, in deductive reasoning:

"If all premises are true, the terms are clear, and the rules of deductive logic are followed, then the conclusion reached is necessarily true." (Source)

So Does Sherlock Holmes Use Inductive or Deductive Reasoning?

Sherlock Holmes is famous for using his deductive reasoning to solve crimes. But really, he mostly uses inductive reasoning. Now that we've gone through what inductive and deductive reasoning are, we can see why this is the case.

Let's say Sherlock Holmes is called in to work a case where a woman was found dead in her bed, under the covers, and appeared to be sleeping peacefully. There are no footprints in the carpet, no obvious forced entry, and no immediately apparent signs of struggle, injury, and so on.

Sherlock observes all this as he looks in, and then enters the room. He walks around the crime scene making observations and taking notes. He might talk to anyone who lives with her, her neighbors, or others who might have information that could help him out.

Then, once he has all the info he needs, he'll come to a conclusion about how the woman died.

That pretty clearly sounds like an inductive reasoning process to me.

Now you might say - what if Sherlock found the "smoking gun" so to speak? Perhaps this makes his arguments and process seem more deductive.

But still, remember how he gets to his conclusions: starting with observations and evidence, processing that evidence to come up with a hypothesis, and then forming a theory (however strong/true-seeming) about what happened.

How to Use Inductive and Deductive Reasoning Together

As you might be able to tell, researchers rarely just use one of these methods in isolation. So it's not that deductive reasoning is better than inductive reasoning, or vice versa – they work best when used in tandem.

Often times, research will begin inductively. The researcher will make their observations, take notes, and come up with a theory that they want to test.

Then, they'll come up with ways to definitively test that theory. They'll perform their tests, sort through the results, and deductively come to a sure conclusion.

So if you ever hear someone say "I deduce that x happened", they better make sure they're working from facts and not just observations. :)

TL;DR: Inductive vs Deductive Reasoning – What are the Main Differences?

Inductive reasoning:

• Based on observations, conversations, stuff you've read
• Starts with information/evidence and works towards a broader theory
• Arguments can be strong and cogent, but never valid or sound (that is, certain)
• Premises can all be true, but conclusion doesn't have to be true

Deductive reasoning:

• Based on testing a theory, narrowing down the results, and ending with a conclusion
• Starts with a broader theory and works towards certain conclusion
• Arguments can be valid/invalid or sound/unsound, because they're based on facts
• If premises are true, conclusion has to be true

And here's a cool and helpful chart if you're a visual learner:

Image source

That's about it!

Now, if you need to conduct some research, you should have a better idea of where to start – and where to go from there.

Just remember that induction is all about observing, hypothesizing, and forming a theory. Deducing is all about taking that (or any) theory, boiling it down, and testing until a certain conclusion(s) is all that remains.

Happy reasoning!

Lookup Bogosort, Slowsort, or Stooge sort on Wikipedia sometime and prepare to enter a deep rabbit hole which leads to the strange world of “perversely awful” sorting algorithms.

These three relatively tame algorithms, used primarily for illustrative purposes in computer science classes, are only the beginning. A strange…

One thing I like the most about being a developer is learning. But to learn something, you must admit that you do not know it.

And sometimes it can be difficult.

Some time ago, I was asked to deal with a linear regression. I was panicked. I had no real idea of what it was about except that it was mathematics. The problem is that I was convinced I should have known that.

I was blaming myself: “But why do you have a baccalaureate (high school degree) in literature instead of a normal baccalaureate like everyone else in your profession would? And why have you preferred studying literature rather than computer science at university? If you had chosen normal studies, you would have known what is a linear regression. You would have known how to deal with that. You have no right to be here. You are an impostor”.

Then I did what I do when I am panicked: I called my sister. She has a “normal” baccalaureate.

“Hi little sister, can you explain me what is a linear regression please?”

She answered me that she had no idea what it was and that she never learned that in high school.

I was relieved. I still did not know what a linear regression was, but I could confess without shame my ignorance. I took some time to understand it and finally dealt with it.

You could say that this story ends well. I am not so sure because I should have confessed my ignorance before calling my sister.

Sometimes, I have the impression that I am not in my place. I feel I am stealing the job of someone else, someone with “normal” studies. Consequently, I am afraid of being exposed.

I know that I am not the only one to get this feeling of being an impostor. There are many reasons you can find yourself in this situation.

I was recently working with a brilliant person with many capabilities. But she was sure of being an impostor because she did not know this framework or this thing in Java, leading her into trouble at work.

If we step back, there is no shame if you do not know something. The most important thing is to admit it.

First, it is impossible to know everything.

Then, if you have not learned something, it is probably because you were doing something else instead. If I did not know until now how Kafka was really dealing with messages from producers, it is because I spent several months learning stuff about Spark. This stuff helped me to pass a certification. That is not so bad in the end.

It was also because I finally recognized that I knew so few things about functional programming with Scala that I decided to follow online courses. Again, that is not so bad.

Saying that you do not know is a way towards progress.

I cannot help thinking about Socrates and this fabulous sentence:

The only thing I know is that I know nothing.

A philosopher — drawing from me

One of the greatest philosophers (for me, as for many) presented himself as ignorant. I believe recognizing yourself as ignorant is a strong way towards self progression and raising the bar.

When you recognize what you do not know, you do not have prejudices. You can learn peacefully. You are not narrow-minded. You are ready to learn everything.

Therefore, I am happy to have learned the concept of linear regression even if at the beginning I was totally lost.

I should have said the truth peacefully and directly: “I know nothing about linear regressions but it is okay. I will learn it”.

Peaceful landscape — picture from Vincent Sebille

Laplace, Bayes, and machine learning today

It may not be a question that you were worrying much about. After all, it appears to happen every day without fail.

But what is the probability the sun will rise tomorrow?

Believe it or not, this question was given consideration by one of mathematics’ all-time greats Pierre-Simon Laplace in his pioneering work of 1814, “Essai philosophique sur les probabilités”.

Fundamentally, Laplace’s treatment of the question was intended to illustrate a more general concept. It was not a serious attempt to estimate whether the sun will, in fact, rise.

In his essay, Laplace describes a framework for probabilistic reasoning that today we recognise as Bayesian.

The Bayesian approach forms a keystone in many modern machine learning algorithms. But the computational power required to make use of these methods has only been available since the latter half of the 20th Century.

(So far, it appears current state-of-the-art AI is keeping quiet on the issue of tomorrow’s sunrise.)

Laplace’s ideas are still relevant today, despite being developed more than two centuries ago. This article will review some of these ideas, and show how they are used in modern applications, perhaps envisaged by Laplace’s contemporaries.

Pierre-Simon Laplace

Born in the small Normandy commune of Beaumont-en-Auge in 1749, Pierre-Simon Laplace was initially marked out to become a theologian.

However, while studying at the University of Caen, he discovered a brilliant aptitude for mathematics. He transferred to Paris, where he impressed the great mathematician and physicist Jean le Rond d’Alembert.

At the age of 24, Laplace was elected to the prestigious Académie des Sciences.

Laplace was an astonishingly prolific scientist and mathematician. Amongst his many contributions, his work on probability, planetary motion, and mathematical physics stand out. He counted figures such as Antoine Lavoisier, Jean d’Alembert, Siméon Poisson, and even Napoleon Bonaparte, as his collaborators, advisers, and students.

Laplace’s “Essai philosophique sur les probabilités” was based upon a lecture he delivered in 1795. It provided a general overview of ideas contained within his work “Théorie analytique des probabilités”, published two years earlier in 1812.

In “Essai philosophique”, Laplace provides ten principles of probability. The first few cover basic definitions, and how to calculate probabilities relating to independent and dependent events.

Principles eight, nine, and ten concern the application of probability to what we might describe today as cost-benefit analysis.

The sixth is an important generalization of Thomas Bayes’ eponymous theorem of 1763.

It states that, for a given event, the likelihood of each possible cause is found by multiplying the prior probability of that cause by a fraction.

This fraction is the probability of the event arising from that particular cause, divided by the probability of the event occurring by any cause.

Probability of A-given-B equals probability of A times probability of B-given-A, divided by probability of B

This theorem’s influence within machine learning cannot be overstated.

The seventh principle is the one that has caused the most controversy since its publication. However, the actual wording is innocuous enough.

Rather, it is Laplace’s choice of discussing the probability of the sun rising the next day by way of illustrative example that has in turn drawn derision and objection over the following two centuries.

The rule of succession is still used today under various guises, and sometimes in the form Laplace originally described.

In fact, the rule of succession represents an important early step in applying Bayesian thinking to systems for which we have very limited data and little or no prior knowledge. This is a starting point often faced in modern machine learning problems.

Laplace’s rule of succession

The seventh principle of probability given in Laplace’s “Essai philosophique” is, in essence, straightforward.

It states that the probability of a given event occurring is found by summing the probability of each of its potential causes multiplied by the probability of that cause giving rise to the event in question.

Laplace then proceeds to outline an example based upon drawing balls from urns. So far, so good. Nothing contentious yet.

However, he then describes how to proceed with estimating the probability of an event occurring in situations where we have limited (or indeed no) prior knowledge about what that probability might be.

“On trouve ainsi qu’un événement étant arrivé de suite un nombre quelconque de fois , la probabilité qu’il arrivera encore la fois suivante est égale à ce nombre augmenté de l’unité, divisé par le même nombre augmenté de deux unités.”

Which translates in English: “So, one finds for a event which has occurred any number of times until now, the probability it will occur again the next time is equal to this number increased by one, divided by the same number increased by two”.

Or, in math notation:

That is, given s successes out of n trials, the probability of success on the next trial is approximately (s+1)/(n+2).

To make his point, Laplace doesn’t hold back:

“… par exemple, remonter la plus ancienne époque de l’histoire à cinq mille ans, ou à 1,826,213 jours, et le soleil s’étant levé constamment dans cet intervalle, à chaque révolution de vingtquatre heures, il y a 1,826,214 à parier contre un qu’il se lèvera encore demain”

Which translates as: “…for example, given the sun has risen every day for the last 5000 years — or 1,826,213 days — the probability it will rise tomorrow is 1,826,214 / 1,826,215”.

At 99.9%, that’s a pretty certain bet. And it only becomes more certain each day the sun continues to rise.

Yet Laplace acknowledges that, for someone who understands the mechanism by which the sun rises and sees no reason why it should cease to function, even this probability is unreasonably low.

And it turns out that this qualification is perhaps just as important as the actual rule itself. After all, it hints at the fact that our prior knowledge of a system is encoded in the assumptions we make when assigning probabilities to each of its potential outcomes.

This is true in machine learning today, especially when we try learning from limited or incomplete training data.

But what is the rationale behind Laplace’s rule of succession, and how does it live on in some of today’s most popular machine learning algorithms?

Nothing is impossible?

To better understand the significance of Laplace’s rule, we need to consider what it means to have very little prior knowledge about a system.

Say you have one of Laplace’s urns, which you know to contain at least one red ball. You know nothing else about the contents of the urn “system”. Perhaps it contains many different colors, perhaps it only contains that one red ball.

Draw one ball from the urn. You know the probability that it will be red is greater than zero, and either less than or equal to one.

But, as you don’t know whether the urn contains other colors, you cannot say the probability of drawing red certainly equals one. You simply cannot rule out any other possibility.

So, how do you estimate the probability of drawing a red ball from the urn?

Well, according to Laplace’s rule of succession, you can model drawing a ball from the urn as a Bernoulli trial with two possible outcomes: “red” and “not-red”.

Before we’ve drawn anything from the urn, we’ve already allowed for two potential outcomes to exist. In so doing, we’ve effectively “pseudo-counted” two imaginary draws from the urn, observing each outcome once.

This gives each outcome (“red” and “not-red”) a probability of 1/2.

As the number of draws from the urn increases, the effect of these pseudo-counts becomes less and less important. If the first ball drawn is red, you update the probability of the next one being red to (1+1)/(1+2) = 2/3.

If the next ball is red, the probability updates to 3/4. If you keep drawing red, the probability reaches ever closer to 1.

In today’s language, probability concerns a sample space. This is a mathematical set of all possible outcomes for a given “experiment” (a process that selects one of the outcomes).

Probability was put on a formal axiomatic basis by Andrey Kolmogorov in the 1930s. Kolmogorov’s axioms make it easy to prove that a sample space must contain at least one element.

Kolmogorov also defines probability as a measure that returns a real valued number between zero and one for all elements of the sample space.

Naturally, probability makes a useful way to model real world systems, especially when you assume complete knowledge about the contents of the sample space.

But when we don’t understand the system at hand, we don’t know the sample space — apart from that it must contain at least one element. This is a common starting point in many machine learning contexts. We have to learn the contents of the sample space as we go.

Therefore, we ought to allow the sample space to contain at least one extra, catch-all element — or, if you like, the “unknown unknown”. Laplace’s rule of succession tells us to assign the “unknown unknown” a probability of 1/n_+_2, after n repeated observations of known events.

Although in many cases it is convenient to ignore the possibility of unknown unknowns, there are epistemological grounds for always allowing such eventualities to exist.

One such argument is known as Cromwell’s Rule, coined by the late Dennis Lindley. Quoting the 17th century’s Oliver Cromwell:

“I beseech you, in the bowels of Christ, think it possible that you may be mistaken”

This rather dramatic statement asks us to allow a remote possibility for the unexpected to occur. In the language of Bayesian probability, this amounts to requiring us to always consider a non-zero prior.

Because if your prior probability is set to zero, no amount of evidence will ever convince you otherwise. After all, even the strongest evidence to the contrary will still yield a posterior probability of zero, when multiplied by zero.

Objections, and a defence of Laplace

It may come as little surprise to learn that Laplace’s sunrise example attracted much criticism from his contemporaries.

People objected to the perceived simplicity — naivety, even — of Laplace’s assumptions. The idea that there was a 1/1,826,215 probability the sun would not rise the following day seemed absurd.

It is tempting to believe that, given a large number of trials, a non-zero probability event must happen. And therefore, observing so many consecutive sunrises without a single failure surely implies Laplace’s estimate is an overestimate?

For example, you might expect that after a million trials, you’d have observed a one-in-a-million event — almost guaranteed by definition! What’s the probability of doing otherwise?

Well, you wouldn’t be astonished if you tossed a fair coin twice without landing heads. Nor would it be cause for concern if you rolled a die six times, and never saw the number six. These are events with probability 1/2 and 1/6 respectively, but that absolutely does not guarantee their occurrence in the first two and six trials.

A result attributed to Bernoulli back in the 17th Century finds the limit as the probability 1/n and number of trials n grow very large:

Although on average you will have observed at least one occurrence of an event with probability 1/n after n trials, there is still a greater than 1/3 chance you will not.

Likewise, if the true probability of the sun failing to rise were indeed 1/1,826,215, then we perhaps shouldn’t be so surprised such an occurrence has never been recorded in history.

And, arguably, Laplace’s qualification is too generous.

It is true that, for a person who claims to understand the mechanism by which the sun rises every day, the probability of it failing to do so must be much closer to zero.

Yet to assume an understanding of such a mechanism requires us to possess prior knowledge of the system, beyond that which we have observed. This is because such a mechanism is implicitly assumed constant — in other words, true for all time.

This assumption lets us, in a sense, “conjure up” an unlimited number of observations — on top of those we have actually observed. It’s an assumption called for by none other than Isaac Newton, at the beginning of the third book in his famous “Philosophiae Naturalis Principia Mathematica”.

Newton outlines four “Rules of Reasoning in Philosophy”. The fourth rule claims we can regard propositions derived from previous observations as “very nearly true”, until contradicted by future observations.

“In experimental philosophy we are to look upon propositions collected by general induction from phenomena as accurately or very nearly true”

Such an assumption was crucial for the scientific revolution, despite being a kick in the teeth for philosophers such as David Hume, who famously argued for the problem of induction.

It is this epistemological compromise that lets us do useful science and, in turn, invent technology. Somewhere along the line, as we see the estimated probability of the sun failing to rise diminish ever closer to zero, we allow ourselves to “round down” and claim a fully fledged scientific truth.

But all of this presumably lies beyond the scope of the point Laplace originally sought to make.

Indeed, his choice of a sunrise example is unfortunate. The rule of succession really comes into its own when applied to completely unknown “black-box” systems for which we have zero (or very few) observations.

This is because the rule of succession offers an early example of a non-informative prior.

How to assume as little as possible

Bayesian probability is a keystone concept in modern machine learning. Algorithms such as Naive Bayes classification, Expectation Maximisation, Variational Inference and Markov Chain Monte Carlo are amongst the most popular in use today.

Bayesian probability generally refers to an interpretation of probability where you update your (often subjective) belief in the light of new evidence.

Two key concepts are prior and posterior probabilities.

Posterior probabilities are those we ascribe to after updating our beliefs in the face of new evidence.

Prior probabilities (or ‘priors’) are those we hold to be true before seeing new evidence.

Data scientists are interested in how we assign prior probabilities to events in the absence of any previous knowledge at all. This is a typical starting point for many problems in machine learning and predictive analytics.

Priors can be informative, in the sense they come with “opinions” about the probability of different events. These “opinions” can be strong or weak, and are usually based on past observations or otherwise reasonable assumptions. These are invaluable in situations where we want to train our machine learning model quickly.

However, priors can also be non-informative. This means they assume as little as possible about the respective probabilities of an event. These are useful in situations where we want our machine learning model to learn from a blank state.

So we must ask: how do you measure how “informative” a prior probability distribution is?

Information theory provides an answer. This is a branch of mathematics that concerns how information is measured and communicated.

Information can be thought of in terms of certainty, or a lack thereof.

After all, in an everyday sense, the more information you have about some event, the more certain you are about its outcome. Less information equates to less certainty. This means that information theory and probability theory are inextricably linked.

Information entropy is a fundamental concept in information theory. It serves as a measure of the uncertainty inherent to a given probability distribution. A probability distribution with high entropy is one for which the outcome is more uncertain.

Perhaps intuitively, you can reason that a uniform probability distribution — a distribution for which each event is equally likely — has the highest possible entropy. For example, if you flipped a fair coin and a biased coin, which outcome would you be least certain about?

Information entropy provides a formal means of quantifying this, and if you know some calculus, you can check out the proof here.

So the uniform distribution is, in a very real sense, the least informative distribution possible. And for that reason, it makes an obvious choice for an uninformative prior.

Perhaps you’ve spotted how Laplace’s rule of succession effectively amounts to using a uniform prior? By adding one success and one failure before we’ve even observed any outcomes, we’re using a uniform probability distribution to represent our “prior” belief about the system.

Then, as we observe more and more outcomes, the weight of the evidence increasingly overpowers the prior.

Case study: Naive Bayes classification

Today, Laplace’s rule of succession is generalised to additive smoothing and pseudo-counting.

These are techniques which allow us to use non-zero probabilities for events not observed in training data. This is an essential part of how machine learning algorithms are able to generalize when faced with inputs not seen previously.

For instance, take Naive Bayes classification.

This is a simple yet effective algorithm that can classify textual and other suitably tokenized data, using Bayes’ theorem.

The algorithm is trained on a corpus of pre-classified data, in which each document consists of a set of words or “features”. The algorithm begins by estimating the probability of each feature, given a certain class.

Using Bayes’ theorem (and some very naive assumptions about feature independence), the algorithm can then approximate the relative probabilities of each class, given the features observed in a previously unseen document.

The best estimate is the class, k, which maximizes the likelihood of observing the individual features, x, within a given document

An important step in Naive Bayes classification is estimating the probability of a feature being observed within a given class. This can be done by calculating the frequency at which the feature is observed in each of that class’s records in the training data.

For instance, the word “Python” might appear in 12% of all documents classed as “programming”, compared to 1% of all documents classed as “start-up”. The word “learn” might appear in 10% of programming documents and 20% of all start-up documents.

Take the sentence “learn Python”.

Using these frequencies, we find the probability of the sentence being classed as “programming” equals 0.12 ×0.10 = 0.012, and the probability of it being classed as “start-up” is 0.01×0.20 = 0.002.

Therefore, “programming” is the more likely of these two classes.

But this frequency-based approach runs into trouble whenever we consider a feature which never occurs in a given class. This would mean it has a frequency of zero.

Naive Bayes classification requires us to multiply probabilities, but multiplying anything by zero will, of course, always yield zero.

So, what happens if a previously unseen document does contain a word never observed in a given class in the training data? That class will be deemed impossible — no matter how frequently every other word in the document occurs in that class.

Additive smoothing

An approach called additive smoothing offers a solution. Instead of allowing for zero frequencies, we add a small constant to the numerator. This prevents unseen class/feature combinations from derailing the classifier.

When this constant equals one, additive smoothing is the same as applying Laplace’s rule of succession.

As well as Naive Bayes classification, additive smoothing is used in other probabilistic machine learning contexts. Examples include problems in language modelling, neural networks, and hidden Markov models.

In mathematical terms, additive smoothing amounts to using a beta distribution as a conjugate prior for carrying out Bayesian inference with binomial and geometric distributions.

The beta distribution is a family of probability distributions defined over the interval [0,1]. It takes two shape parameters, α and β. Laplace’s rule of succession corresponds to setting α = 1 and β = 1.

As discussed above, the beta(1,1) distribution is the one for which information entropy is maximised. However, there are alternative priors for cases in which the assumption of one success and one failure are not valid.

For instance, Haldane’s prior is defined as a beta(0,0) distribution. It applies in cases when we are not even sure if we can allow for a binary outcome. Haldane’s prior places an infinite amount of “weight” on zero and one.

Jeffrey’s prior, the beta(0.5, 0.5) distribution, is another non-informative prior. It has the helpful property that it remains invariant under reparameterization. Its derivation is beyond the scope of this article, but if you are interested, check out this thread.

The legacy of ideas

Personally, I find it fascinating how some of the earliest ideas in probability and statistics have survived years of contention, and still find widespread use in modern machine learning.

It is extraordinary to realise that the influence of ideas developed more than two centuries ago is still being felt today. Machine learning and data science have gained real mainstream momentum in the last decade or so. But the foundations upon which they are built were laid long before the first computers were even close to realization.

It’s no coincidence that such ideas border on the philosophy of knowledge. This becomes especially relevant as machines become more and more intelligent. At what point might the focus shift onto our philosophy of consciousness?

Finally, what would Laplace and his contemporaries make of machine learning today? It’s tempting to suggest they’d be astounded by the progress that’s been made.

But that would probably be a disservice to their foresight. After all, the French philosopher René Descartes had written of a mechanistic philosophy back in the 17th Century. Describing a hypothetical machine:

“Je désire que vous considériez … toutes les fonctions que j’ai attribuées à cette machine, comme … la réception de la lumière, des sons, des odeurs, des goûts … l’empreinte de ces idées dans la mémoire … et enfin les mouvements extérieurs … qu’ils imitent le plus parfaitement possible ceux d’un vrai homme … considériez que ces fonctions … de la seule disposition de ses organes, ni plus ni moins que font les mouvements d’une horloge … de celle de ses contrepoids et de ses roues”

Which translates as: “I desire that you consider that all the functions I’ve attributed to this machine such as… the reception of light, sound, smell and taste… the imprint of these ideas in the memory… and finally the external movements which imitate as perfectly as possible those of a true human…Consider that these functions are only under the control of the organs, no more or less than the movements of a clock are to its counterweights and wheels”

The passage above describes a hypothetical machine capable of responding to stimuli and behaving like a “true human”. It was published in Descartes’ 1664 work “Traité de l’homme” — a full 150 years before Laplace’s “Essai philosophique sur les probabilités”.

Indeed, the 18th and early 19th Centuries saw the construction of incredibly sophisticated automata by inventors such as Pierre Jaquet-Droz and Henri Maillardet. These clockwork androids could be “programmed” to write, draw, and play music.

_Drawing produced by one of Henri Maillardet’s early 19th Century automata ([video](https://www.youtube.com/watch?v=LCNQvm61_78" rel="noopener" target="blank" title="))

So there is no doubting that Laplace and his contemporaries could conceive of the notion of an intelligent machine. And it surely would not have escaped their notice how progress made in the field of probability might be applied to machine intelligence.

Right at the beginning of “Essai philosophique”, Laplace writes of a hypothetical super-intelligence, retrospectively named “Laplace’s Demon”:

“Une intelligence qui, pour un instant donné, connaîtrait toutes les forces dont la nature est animée, et la situation respective des êtres qui la composent, si d’ailleurs elle était assez vaste pour sou- mettre ces données à l’analyse … rien ne serait incertain pour elle, et l’avenir comme le passé, serait présent à ses yeux”

Which translates as: “An intelligence, which in a given moment, knows all the forces by which nature is animated, and the respective situation of the beings which compose it, and if it were large enough to submit these data to analysis … nothing would be uncertain to it, and the future as the past, would be present in its eyes”.

Could Laplace’s Demon be realized as one of Descartes’ intelligent machines? Modern sensibilities overwhelmingly suggest no.

Yet Laplace’s premise envisaged on a smaller scale may soon become a reality, thanks in no small part to his own pioneering work in the field of probability.

Meanwhile, the sun will (probably) continue to rise.

Functional reactive programming is laden with unfamiliar terminology to the newcomer: pure functions, immutability, monads, etc. But beneath these concepts is a deeper principle — one debated long before Charles Babbage and the first computers. I argue the difference between object-oriented programming (OOP) and functional reactive programming (FRP) is as much about interpretations of reality as it is about structures of programs.

Here’s a thought experiment we’ve all likely heard:

If a tree falls in the middle of a forest, and no one is there to hear it, does it make a sound?

There are many of ways to attack this question as ill-posed. Ignoring them for the moment, this question is poking at a fundamental aspect of reality: causality. Is existence dependent on, or independent of, observation? Let’s translate this thought experiment into code. Here’s a tree:

class Tree {     constructor(){         this._fell = false;     }     set fell(state){         this._fell = state;     }     get fell(){         return this._fell;     }}

var tree = new Tree();tree.fell = true;

To make the tree fall we set its fallen state to true. This is textbook object-oriented programming. Its patterns are getters, setters, and state. Simple enough, but in the context of our thought experiment there is a lurking interpretation: if one questions whether the tree fell, even if they didn’t observe it, the answer is a resounding “Yes!” One only needs to check that tree.fell is true. Those that answer yes to our philosophical question do so because they can return to the forest, see the fallen tree, and deduce it fell in the past. Here is the codified equivalent. Looks like we’ve solved that centuries-old riddle.

Not so fast! Let’s look at a different approach. Here’s another tree:

class Tree extends EventEmitter {}var tree = new Tree();tree.emit('fall');

This is the “reactive” tree. Its patterns are events and transforms. In its purest form our tree maintains no state. We make it fall by broadcasting a fall event. Alas, the message falls on deaf ears! No state has changed, no evidence is left. There is no way to deduce the past by querying reality. Did the tree fall? shrug

Descartes and Berkeley

The object-oriented and reactive approaches give two different answers to our thought experiment because they embody two contradictory philosophies of epistemology: Rationalism, popularized by Descartes in the late 1600s, and Empiricism popularized by Berkeley in the early 1700s.

Descartes takes as a short break from writing a Java interface

Descartes, in a streak of fanatical skepticism, found he could only be sure of one thing: his own existence. He came to this conclusion because he couldn’t doubt the existence of his thoughts and concluded there must be some entity doing the thinking, thus coining the phrase, cogito ergo sum: I think therefore I am. In other words: by acknowledging that there is some internal state that is changing, there must be some agent for whom the state belongs. To Descartes, changes of state is proof of existence — just like our first tree.

Berkeley seen pondering Haskell type definitions

Soon after Descartes comes George Berkeley. Berkeley denounced the realist’s view. To Berkeley, it made no sense for material objects, like trees, to have existence. Existence only comes to us through thoughts (mental as opposed to physical experience), and thoughts must assimilate in the mind to exist. Material objects are deceptions; their essence is not their physicality but their ability to transform the immaterial. If a thought is not assimilated in the mind, it has no existence. Thus he popularized the Latin phrase esse percepi: to be is to be perceived.

Let’s translate Berkeley’s reality into code. For our second tree to make a sound a mind must interpret it. We will create a chain of causality, starting from the tree falling, to the air vibrating, to the ear creating an electrical stimulus, to the brain interpreting it as sound.

When the tree falls, the air vibrates.

class Air extends EventEmitter {    constructor (){        super();                function mapFall (fall){...}        this.on('fall', (fall) => {            var vibration = mapFall(fall);            this.emit('vibrate', vibration);        };    }}

When the air vibrates, the ear converts it to an electrical stimulus.

class Ear extends EventEmitter {    constructor (){         super();                function mapFrequency (frequency){...}        this.on('vibrate', (frequency) => {            var stimulus = mapFrequency(frequency);            this.emit('stimulus', stimulus);        };    }}

When ear creates a stimulus, the brain interprets it as sound.

class Brain extends EventEmitter {    constructor (){        super();        function mapStimulus (signal){...}        this.on('stimulus', (signal) => {            var sound = mapStimulus(signal);            this.emit('sound', sound);        };    }}

We have effectively set up a chain of causality, which we make explicit by building a pipeline:

tree.pipe(air).pipe(ear).pipe(brain);

Now, when the tree falls it makes an impression on a mind:

brain.on('sound', (sound) => {    // We exit the system. You have been heard!    console.log(sound); });

tree.emit('fall', fallData);

Berkeley called this concept Subjective Idealism. Idealism because it asserts only thoughts or ideas exist, and subjective because reality is dependent on the subjects that perceive it. In my opinion, Subjective Idealism is the philosophy underpinning reactive programming. Berkeley writes,

It is indeed an opinion strangely prevailing amongst men, that houses, mountains, rivers, and in a word all sensible objects have an existence natural or real, distinct from their being perceived by the understanding. …For what are the fore-mentioned objects but the things we perceive by sense…and is it not plainly repugnant that any one of these or any combination of them should exist unperceived?

I love this quote for its self-assured audacity. Berkeley is essentially calling us crazy for thinking that houses, mountains and rivers exist. In our example, trees, air, ears and brains are false idols; the only reality is mapFall, mapFrequency and mapStimulus. Reality is then never consumed, as it is with objects when they retain state. Reality is merely transformed.

Subjective Idealism in Practice

In OOP we create objects that encapsulate some kind of behavior. We then construct programs which are networked relationships of these objects. Our program is structurally a graph.

In FRP we create pipelines of functions that encapsulate causal relationships. Pipelines are then merged and branched to give the graph-like structure of an object-oriented program. However, there is an important limitation on the types of functions. Only pure functions are allowed. That is, functions that cannot effect anything outside themselves. In our example, the Ear object cannot change how the Air object vibrated. This constraint ensures that our pipelines have a well-defined direction from cause to effect. In terms of structure, this means our program is a directed acyclic graph (DAG).

To reason about software, we must think of it as a sequence of causal relationships. We must be able to order the program. Mathematically, a graph can be ordered if and only if it is a DAG. This is true no matter how you write your program. Whether you choose OOP, FRP or XYZ. What’s special about FRP, though, is that ordering is enforced by the pattern.

In OOP, ordering is left unspecified. Objects can call methods on other objects. Objects can change the state of other objects. Everything has read and write privileges by default. Specifying an order is done manually by the developer. It is up to them to relate the sequential ordering of lines in a program to an ordering of the objects’ causal relationships.

Unfortunately, this is all too easy to mess up. Notice that in OOP, when two objects write to the same shared state, you have a race condition. In FRP, when two functions try to affect one another you have an infinite loop. This is but one example of a theoretical pattern enforcing a practical result.

The bottom line is that it is not enough to encapsulate state. A well-written program will also encapsulate dependency.

Tradeoffs

“Programmers know the benefits of everything and the tradeoffs of nothing.”— Rich Hickey

You’d think after all this FRP praising and OOP bashing, I’d be firmly in the FRP camp. You’d be wrong! FRP is a programming pattern, and patterns serve to constrain solutions. If the ideal solution doesn’t satisfy the constraints, you’ll be wasting energy fighting against the pattern.

To be concrete, there are a few simple annoyances of FRP. Take immutability — you are always creating more memory. You can never, say, sort a list in place. You will always be creating a new list. In theory, immutability is a good pattern to observe. In practice, you may be memory constrained, and it may be a good idea to swim against the FRP current.

But this is not the glaring problem. The glaring problem is that FRP becomes an anti-pattern when you don’t know when two parts of a codebase will interact. Take, for example, a first person shooter game. Somewhere is a bullet object, and somewhere else is a player object. A bullet may hit a player, but it’s unclear when this will happen. These objects need to retain state (velocity of the bullet, health of the player, etc.) so it is available at the moment they interact. Perhaps in the abstract the entire game can be thought of as one causal pipeline, but that sounds more daunting to me than thinking in terms of decoupled objects and state.

To put on my philosophical hat once more, physics may decree that reality is one causal pipeline whose time evolution is governed by deterministic physical laws, and whose initial conditions (or probabilities) are provided by the big bang. But this is hardly how humans reason about cause and effect. We naturally slice up reality into higher-level objects and reason about their inter-relations. It can be simpler that way, even though it’s error prone!

I feel this is why FRP hasn’t been been wholly embraced by the programming community, even after seeing its many advantages. The best we should hope for when writing programs is to use FRP principles in places where its patterns fit the solution, and let them inspire OOP patterns where its patterns do not. To me this is a distinction between solutions that can be thought of as pipelines, and solutions that shouldn’t be.

Conclusion

In philosophy the objective is not to solve our deepest problems, but to have a shared language and historical precedent to debate them. So when a new problem emerges, we don’t have to start over. Similarly for programming patterns. They are not used to decide right and wrong, as if these concepts have universal appeal. They are used to classify problems and their approaches.

We should also look to other disciplines, much older than computer science, to see whether their shared language and historical precedent can inspire our own. Then we may see that the question, “did the tree fall?” is not answered with a yes or a no. That instead it questions a perspective. One that can frame our philosophy of epistemology or our architecture of programs. And one to which the answer lies between a state of mind, and a flow of thought!

Preview of Part II

There is still a nagging dilemma in our thought experiment. Even if there are no people to experience the sound, surely when the tree falls the air must vibrate! Right? In Part II we will go further down the rabbit hole to see that FRP can give a surprising interpretation here too. We’ll think of data flows in terms of push vs pull and see how this appears in concepts like lazy evaluation, algorithms like ray tracing, and applications like Excel.

Computer programs are all around us. We interact with them every day. It looks as if software is becoming more and more important to our society. But why do we find programs so necessary to us? Why and when did we start programming? What is the essence of programming? These questions might sound trivial, but I feel that today we still don’t have a good definition of what programming is. Maybe this post can help to change that.

It is pretty hard to define programming because it is so diverse. One can program games, mobile apps, websites, compilers, simulations and much more. In this case it might be helpful to start by breaking down some misconceptions and clarifying what programming is not.

Programming is not science. Science is the art of examining the world and discovering repeatable patterns in it. The scientific method is about making a hypothesis, and then performing experiments to prove or reject it. We don’t use such a method in programming — thus it is not science. Programming is not about discovery, but creativity.

Programming is not math. Yes, there is an aspect of math to programming. Some parts of programs can be expressed as mathematical functions. Writing a Fibonacci number generator is fun, but completely useless without a real-world application. Math in programming is means to an end, not a core part of the process.

Programming is not about electronic computers. Computers are very useful, but not necessary. Programs can be understood and interpreted by human beings as well. Therefore, computers are just tools we use in programming.

So what is programming about? What is at the core of every program — big or small?

Abstractions.

What is abstraction? It is a reduced image of the world. When abstracting, we are converting reality into symbols which can be transmitted as information. The word abstraction originates from two Latin words, which are abs, meaning away from and trahere, meaning to draw. The Latin translation suggests that abstracting implies separating one from the whole.

Is the process of abstraction unique to programming? Not really. It is something that humans have been doing for quite some time now. And our most basic tools of abstraction are the map and the clock.

Archaeologists have discovered stone maps that humans made more than 14,000 years ago. This shows that map-making is fundamental to humans. It is a process of transforming territory into symbols by abstracting it. A map is a way to abstract space. It is a tool that helps us to make sense of surrounding territory so that navigating it will be easier.

A clock, on the other hand, is a way to abstract time. We find the continuous nature of time confusing, so we abstract it. Humans split time into discrete intervals: years, months, days, hours, minutes, seconds. While a map helps us navigate space, a clock helps us navigate time. The predecessor of the clock — the calendar — emerged more than 10,000 years ago.

_Giant solar calendar [Chankillo](https://en.wikipedia.org/wiki/Chankillo" rel="noopener" target="blank" title=") was built 2,300 years ago.

What about computers then? They are also tools that deal with abstractions. There are 3 fundamental parts to every computer:

1. Internal clock — a computer’s way to abstract time
2. Memory — a computer’s way to abstract space
3. Processing unit —a computer’s way to perform logical operations

These facilities give computers a way to understand space abstractions and their interactions in abstracted time. This means that programs are abstractions of spacetime, and programming is the art of creating spacetime abstractions. These abstractions help us to navigate reality, and that is why they are so important to us.

And there is one abstraction method that is very similar to programming, but is even older than maps, clocks and calendars: language.

If you look closer at our natural language, you will see that it has all the features necessary to abstract spacetime.

Let’s look at an example sentence:

Go to the garden and pick up some flowers this evening.

The garden and flowers refer to abstracted space. This evening is a way to abstract time. And adds logic to the sentence. Go to and pick up are subroutines.

We can easily transform the sentence above into JavaScript:

whenEvening.then(()=>you.goTo(garden)).then(()=>you.pickUp(flower))

This sentence can be understood by a computer that has definitions of evening, you, flowers, and the necessary subroutines defined.

The problem with natural language is that it has very broad applications. Language can be used not only to communicate information, but also to express feelings and emotions. The best examples of pure natural language programs are laws, board game rules, and instruction manuals.

_[ CC BY-NC-SA](https://en.wikipedia.org/wiki/Code_of_Hammurabi" rel="noopener" target="_blank" title="">Code of Hammurabi is one of the oldest natural language programs (photo by Richard /<a href="https://creativecommons.org/licenses/by-nc-sa/2.0/" rel="noopener" target="blank" title="))

Programming languages, on the other hand, are strict and can only create abstractions. Computers are designed to interpret these abstractions in a very specific and deterministic way.

In order to write computer programs one has to learn how to encode natural language to symbols that computer can understand. This usually requires intricate knowledge of computer architecture and the syntax of the chosen computer language. Thus if you want a computer to understand your reality abstractions, you have to learn how to code.

Computers are tools that can execute rules defined in programs with superhuman speed and precision. They empower us with the ability to construct complex, multilayered abstractions, and transform our programs into truly dynamic and interactive maps of reality.

In many ways, programming is something that all of us already know how to do. The process of reality abstraction is fundamental to human beings. Computer programming is just the most effective way of doing this.

Alan J. Perlis was the first recipient of the Turing Award. He’s possibly most remembered today for his Epigrams on Programming, which are still brilliant and instructive more than three decades on.

One of my favorite quotes not from the Epigrams is about staying engaged in the actual business of learning, practicing, and — most importantly—enjoying computer programming.

I hope the field of computer science never loses its sense of fun. Above all, I hope we don’t become missionaries. Don’t feel as if you’re Bible salesmen. The world has too many of those already. What you know about computing other people will learn. Don’t feel as if the key to successful computing is only in your hands. What’s in your hands, I think and hope, is intelligence: the ability to see the machine as more than when you were first led up to it, that you can make it more.

— Alan Jay Perlis

Here are a few of his epigrams to whet your appetite:

7. It is easier to write an incorrect program than understand a correct one.

8. Recursion is the root of computation since it trades description for time.

9. There will always be things we wish to say in our programs that in all known languages can only be said poorly.

10. Programmers are not to be measured by their ingenuity and their logic but by the completeness of their case analysis.

11. Beware of the Turing tar-pit in which everything is possible but nothing of interest is easy.

12. Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it.

13. In programming, as in everything else, to be in error is to be reborn.

14. When someone says “I want a programming language in which I need only say what I wish done,” give him a lollipop.

15. Don’t have good ideas if you aren’t willing to be responsible for them.

16. Dealing with failure is easy: Work hard to improve. Success is also easy to handle: You’ve solved the wrong problem. Work hard to improve.

If you happen to use WordPress and want to read some more Perlis right in your dashboard, check out my plugin Hello Perlis, adapted from Matt Mullenweg’s Hello Dolly plugin.